optimized noninvasive monitoring of thermal changes on digital b-mode renal sonography during...

8/14/2019 Optimized Noninvasive Monitoring of Thermal Changes on Digital B-Mode Renal Sonography During Revasculariza

1/13

Optimized Noninvasive Monitoring ofThermal Changes on Digital B-ModeRenal Sonography DuringRevascularization Therapy

Objective. Noninvasive real-time thermal change monitoring of human internal organs can play a crit-

ical role in diagnosis and treatment of many disorders, including reperfusion of renal arteries during

anticoagulation therapy. Methods. This article focuses on tissue temperature detection using ultra-

sound velocity changes in different structures and their related speckle shift from their primary loca-

tions on high-quality B-mode digital sonography. We evaluated different speckle-tracking techniques

and optimized them using appropriate motion estimation methods to determine the best algorithm

and parameters. Results. Performing thermal detection methods on simulated phantoms showed a

good correlation between speckle shifts and the ground truth temperature. For the simulated images,

average thermal error was 0.5C with an SD of 0.5C, where lower errors can be obtained in noiseless

(motionless) data. The proposed technique was evaluated on real in vivo cases during surgical occlu-

sion and reopening of the renal segmental artery and showed the potential of the algorithm for obser-

vation of internal organ changes using only digital ultrasound systems for diagnosis and therapy.

Conclusions. The adaptive Rood pattern search proved to be the best block-matching technique,

whereas the multiresolution Horn-Schunck technique was the best gradient optical flow method. The

extracted thermal change during in vivo revascularization therapy is promising. In addition, we present

an evaluation of several block-matching and optical flow motion estimation techniques. Key words:

block matching; optical flow; reperfusion therapy; speckle tracking; ultrasound thermometry.

Received February 4, 2009, from the Department ofBiomedical Engineering, Tehran University ofMedical Sciences, Tehran, Iran (M.D.A.); Departmentof Electrical and Computer Engineering, Universityof Louisville, Louisville, Kentucky USA (V.T.); andScience and Research Branch, Islamic AzadUniversity, Tehran, Iran (N.S.). Revision requested

March 16, 2009. Revised manuscript accepted forpublication April 21, 2009.

We thank Ali Najafian and Hossein Jandaghi forassistance with animal surgery and sonographicimage acquisition and Ehsan Mohammadi for

preparing agar gel phantoms. This work was sup-ported in part by the Research Center for Scienceand Technology in Medicine, Tehran University ofMedical Sciences.

Address correspondence to Vahid Tavakoli, MD,MS, Department of Electrical and ComputerEngineering, University of Louisville, Medical ImageComputing Laboratory, Lutz Hall, Room 308,Louisville, KY 40292 USA.

E-mail: [email protected]

AbbreviationsARPS, adaptive Rood pattern search; ES, exhaustive search;4SS, 4-step search; MAD, mean absolute difference; MSE,mean square error; PSF, point spread function; ROI, regionof interest; 3SS, 3-step search; 2D, 2-dimensional

Tissue Temperature Detection

Thermal changes in different parts of the human body

are directly related to the physiologic characteristics and

functions of the organs. The main supplier of the heat for

any organ is blood flow, and any disturbance in tissue

perfusion can be shown as a temperature gradient. Also,

many nonvascular disorders such as inflammations and

abscess formation are related to the changes in tempera-

ture. Therefore, superficial or mucosal thermal detection

(ordinary thermometer) is a major part of any physicalexamination in any clinic.1 However, for a deeper tem-

perature measurement, there is a lack of a standard tech-

nique. Moreover, ordinary electronic sensors (infrared

detectors and thermistors) are restricted to local use in

specific parts, are implanted, and are also invasive. In this

article, we introduce a novel method for assessing ther-

mal changes using speckle shifts on high-quality B-mode

sonography. This technique is not only noninvasive but

also real time and easily performable on an entire organ

even in 3-dimensional views.2

2009 by the American Institute of Ultrasound in Medicine J Ultrasound Med 2009; 28:15351547 0278-4297/09/$3.50

Technical Advance

Author: Please give degrees for names mentioned in acknowledgments.

Mohammad D. Abolhassani, PhD, Vahid Tavakoli, MD, MS, Nima Sahba, MS


2/13

The proposed technique is used to observe the

thermal changes during renal revascularization

therapy because revascularization (opening an

occluded artery) redirects the blood flow to the

tissue and elevates the temperature.

Thermal Detection Using Sonography

Use of laser and radio frequency knives, which

are based on thermal-controlling applications,

requires accurate thermal detection. Treatment

monitoring consists of real-time tissue damage

monitoring and real-time temperature estima-

tion.3,4Regarding ultrasound thermometry, different

methods have been proposed so far, such as analysis

of the frequency-dependent attenuation,5 speed

of sound, thermal expansion,69 and back-scattered

power.10 One novel technique is ultrasound-simulated acoustic emission, which was shown

to be temperature dependent.11 Konofagou et al12

used ultrasound-simulated acoustic emission in

detection and monitoring of thermal lesion for-

mation and thermal therapy.13,14

For many materials, the ultrasound velocity is

dependent on the temperature. The estimations

of the temperature variations are obtained by

calculating the axial displacement of the scatter-

ers due to tissue warm-up. The displacements

are caused primarily by the changes of the speed

of sound in the tissue when the temperature

changes. The second reason for the changes of

the signals is the thermal expansion of tissue. The

dependency of the speed of sound on the tem-

perature introduces apparent or virtual shifts in

the scatterer positions, and the thermal expan-

sion of the medium introduces physical shifts in

the scatterer positions. The variation of the speed

of sound with respect to the temperature is for-

mulated as Equation 1:

(1)

Where c is the ultrasound velocity; T is the

temperature at distance x; c0

(x) = c(x, T0); and

is the coefficient of speed change

with regard to temperature. Moreover,

(2)

where is a medium-dependent

parameter; is linear coefficient of thermal

expansion; and dis the displacement.

Details of the formulation have been describedpreviously. In a novel method, Abolhassani et al2

proved the correlation between the temperature

estimation and the thermal speckle shifts on

high-quality B-mode digital sonography in tissue-

mimicking phantoms through which a heater

electrode and an array of thermistors were

inserted. The average error of their study was

0.2C; the maximum error was 0.53C; and the SD

was 0.53C.

Speckle-Tracking Methods

To date, there have been many methods ofspeckle tracking between 2 ultrasound images.

Block matching is one of them, which focuses on

the gray scale similarity between 2 sliding blocks

of an image. There are many methods of observ-

ing the similarity between 2 blocks; for instance,

the mean absolute difference (MAD), mean

square error (MSE), maximum likelihood, and

entropy maximization. Moreover, there are sev-

eral approaches to using these block-matching

techniques. An old and famous one is exhaustive

block matching, but its time consumption has

led to newer methods with comparable results,

such as the 3-step search (3SS), 4-step search

(4SS), cross-diamond search, adaptive Rood pat-

tern search (ARPS), and simple and efficient

algorithms. These techniques have been dis-

cussed in detail previously.1523

Another group of motion estimation methods,

introduced in 1981, was the gradient optical flow

methods such as Lucas-Kanade (which pre-

sumes intensity constancy between the consec-

utive frames) and Horn-Schunck (a velocity

constancy assumption is added to the previousmethod). Many other optimizations have been

performed on these techniques, such as phase

matching, discrete cosine transform preprocess-

ing, uniform or gaussian prefiltering, invariant

moment implementation, histogram equaliza-

tion, a multiresolution approach (Suhling-Unser

strategy), and new iterations or warping flow

charts.2325

1536 J Ultrasound Med 2009; 28:15351547

Monitoring of Thermal Changes During Revascularization Therapy


3/13

Moment Implementation and Multiresolution

Strategy

During motion, pixels of an image not only

translate but also shear and rotate. Based on this

fact, many motion estimation methods makeuse of moments (Zernike or spline) to achieve a

high degree of invariance and robustness to

shear and noise.2628

On the other hand, moment implementation is

not the whole story. In each frame, pixels have

different motion vectors in each region; ie, each

region of an image translates, rotates, or shears

into different sizes. To overcome this problem, we

proposed a multiresolution strategy inspired by

the Suhling-Unser multiresolution method.29,30

In the Suhling-Unser technique, coarse kernels

are obtained on the whole image, and then finerkernels are implemented until a certain amount

of error is satisfied. It means that areas of large

motion are computed using coarser windows,

and areas of fine motions are computed using

small moments. Using this technique, we can

capture smaller regions of motion that would be

invisible to a large-kernel method.

From Speckle Tracking to Thermal Change

Detection

Equations 1 and 2 relate the speckle displace-

ment to the thermal change. Each speckle shift

shows its own thermal shift in addition to the

thermal shift of the proximal speckles (with

respect to the probe). Therefore, there is an accu-

mulation effect in the thermal speckle shift on

B-mode ultrasound images.

To recover the thermal change of each speckle,

it has been proven that vertical displacement

derivation is necessary. The derivation step can

enhance the noise, and it is crucial to preserve

the robustness of the proposed method. Some

articles have discussed the noise sensitivity ofthe thermal change detection during the vertical

derivation and showed that an order statistics fil-

ter can extract the thermal changes from the

speckle displacements.31

Handling Motion Artifacts in the z-Axis

Several studies have shown that the optical

flowbased methods can handle large motion arti-

facts in the z-axis. Several studies extracted motion

vectors on 2-dimensional (2D) B-mode echocar-

diographic images from the 3-dimensional heart

motion.3234As will be discussed later, this error is

called an out-of-plane error and can be overcome

using temporal windowing and confidence mea-

surement.

Materials and Methods

In this section, we present different motion esti-

mation techniques that were used in this project.

Several methods were implemented and opti-

mized using different parameters. The algo-

rithms were evaluated on the simulated thermal

change in the simulated tissue to evaluate their

effectiveness and performances.

Motion Estimation Methods for Speckle TrackingThere are many different methods for extracting

motion between different images, such as block-

matching techniques, gradient-based methods

(optical flow), and feature correlation. Figure 1

illustrates different types of motion estimation

methods.

The block-matching methods try to find a sim-

ilar block in 2 frames. Optical flow is inspired by

the block-matching method, whereas both of

them assume intensity constancy in a region or

between several pixels. Nevertheless, optical flow

algorithms use advanced variational calculus

techniques that make them very fast. The feature

matching and correlation techniques extract 2

similar features in 2 frames. These techniques

suffer from loss of features, finding good features

to track, and providing dense motion fields.

However, the optical flow techniques are very

fast, but their sensitivities to noise can induce

inaccurate results.

The best approach to find an efficient motion

estimation technique is the implementation of

different methods on simulated or real images.The simulated images provide us with accurate

measurements, whereas it is usually very difficult

to obtain accurate and dense results from the

real sequences.

To find out an optimal technique for ultra-

sound thermal detection, we implemented dif-

ferent algorithms on the simulated ultrasound

images and then validated the results on the real

ones. Figure 1 shows different types of motion

estimation techniques.

J Ultrasound Med 2009; 28:15351547 1537

Abolhassani et al


4/13

There are several major differences between

echocardiographic series and camera sequences.

Camera movies primarily suffer from occlusion

and aperture and transparency problems. On the

other hand, echocardiographic series mainly sufferfrom an out-of-plane error, a low signal to noise ratio,

and violation of the Nyquist theorem in the spa-

tiotemporal dimension. Echocardiographic motion

estimation is especially dependent on spatiotem-

poral aliasing and an out-of-plane error; there-

fore, higher frame rates can dramatically increases

the accuracy.

Block-Matching Algorithms

The main idea toward this type of motion esti-

mation is the fact that each block of a frame

moves toward a similar block in another frame ifthe time interval is small enough. The general

strategy is to move a microblock over the other

blocks and compute the most similar blocks. To

find the best block matching, it is necessary to

have a similarity-matching function called the

cost function, through which similarity between

consecutive images is extracted. There are sever-

al famous block-matching algorithms including

the MAD and MSE:

(3)

(4)

whereNis the size of the macroblock, and Cand

R define pixel shifting.

Exhaustive Search Method (Full Search Method)

This method computes the cost function of any

block of the macroblock (area in which we arelooking for a smaller block). This is a very time-

consuming but an exact method.

Another variation is the cross-correlation

method. In this algorithm, 1 block of the image

slides over the next image, and the most corre-

lated block to the primary block is considered

a matched block with which motion can be

estimated:

(5)

The maximum peak of c(m, n) indicates the

speckle displacement, where SP1

and SP2

are the

sliding microblock and macroblock, respectively.

The usual correlation methods are computation-

ally expensive, and this problem has led to manyoptimizations in this area, such as gradient

descent-based approaches.

Other Variations of Block Matching

To cope with the computational expense of the

exhaustive search (ES), several searching strategies

are proposed, such as the 3SS, 4SS, simple and effi-

cient search, new 3SS, diamond search, and ARPS.

Gradient-Based (Optical Flow) Methods

Horn-SchunckThe general differential formulation of the opti-

cal flow computation assumes that the intensity

of a particular point in a moving pattern does not

change with time.6 This constant intensity

assumption is written as

(6)

where E is the pixel intensity with the gradient

toward the spatial and temporal dimension, and

is a weighting parameter.23

Locally Constant Motion (Lucas-Kanade)

As an alternative to the brightness constancy

model in Equation 6, the Lucas-Kanade method

(Equation 7) proposed to solve the problem on

small local windows where the motion vector is

assumed to be constant. We ought to minimize

this equation:

(7)

where Eis the pixel intensity, and Wis a weight-

ed vector.

Results will be

ATW2AV = ATW2b ,

where W = diagonal [W(x1),...,W(x

n)]; b = (E

t

(x1),...,(Et (xn)); and u and vare our motion vec-

tors inxandydirections.25

1538 J Ultrasound Med 2009; 28:15351547



5/13

Multiresolution Approach

Primarily coarse kernels are obtained on the

whole image, and then finer kernels are imple-

mented until a certain amount of error is satis-

fied. Therefore, large motions are computedusing coarse windows, and fine motions are com-

puted using small moments. Multiresolution

moment implementation for ultrasound images

has been discussed in detail previously.2630

Motion Extraction in Simulated Data

Renal tissue movement consists of 2 major sub-

group motions: (1) motion artifacts due to respi-

ration and patient displacement and (2) thermal

speckle shift. The steps resulting to the thermal

estimation are shown in Figures 2 and 3.

It is obvious that for accurate motion estima-tion, we ought to remove the first group, which

is more prominent and destructive. B-mode

sonography has a periodic nature because of res-

piratory motion; therefore, noise removal is cru-

cial for obtaining a motionless frame. The

following steps are implemented to extract the

thermal change motion:

1. Region of interest (ROI) where renal tissue is

extracted. The pixels in this area define the renal

tissue, but those outside the ROI pertain to

abdominal organs.

2. Region of interest neighborhood motion esti-

mation. The motion vectors around the ROI area

are extracted, and on the basis of these vectors,

the interpolation is performed in the ROI using a

cubic B-spline to approximate the pure ROI

motion artifact vectors.

3. Warping strategy. The fact that there are

always some degrees of motion in the proximity

of renal tissue, even in carefully fixed frames,

necessitates ROI adjustment. Note that we have

a combination of the motion artifact and the

thermal speckle shift inside the renal tissue.A motion interpolation step using cubic B-spline

interpolation (called Ispline) was implemented

in the previous step with the neighboring renal

pixels (ROI) to extract the motion artifact in the

ROI. After that, the motion artifact vectors adjust

the primary image pixel shifting after a simple

subtraction.

4. After the ROI adjustment, the motion estima-

tion is performed in the ROI. Many speckle-

tracking techniques were implemented on

images to evaluate their efficacy and time con-

sumption in favor of the novel optimized tech-

nique. Shifts in B-mode frames show thermal

velocity changes if motion artifacts are desirably

removed. We implemented block-matchingmethods (3SS, 4SS, cross-diamond search, sim-

ple and efficient search, and ES using the MSE

and MAD) and gradient-based optical flow

methods (Lucas-Kanade, Horn-Schunck, and

iterative Lucas-Kanade with different numbers

of iteration). These methods were performed

using Zernike and spline moments to observe

the accuracy of different sizes, orders, and types.

J Ultrasound Med 2009; 28:15351547 1539

Abolhassani et al

Figure 1. Different types of motion estimation techniques.

Figure 2. Schema of thermal extraction steps.


6/13

Transformation From the Speckle Shift to the

Thermal Change

According to Equation 2, knowing k, the medium

dependent parameter, which can be obtained

experimentally for different tissues, and the pixel

displacement, we can estimate the local temper-

ature change.

Renal B-Mode Sonographic Simulation

We simulated abdominal sonography in the renal

tissue region regarding the gaussian speckle archi-

tecture and randomness. Also, we added respira-

tory motion with respect to the diaphragmatic

motion, modeled as a sinusoidal expansion and

contraction of the abdomen with varying frequen-

cies from 14 to 22 per minute, as in the normal

human body. A small part of these images had

very small changes to simulate thermal speckle

shifts, and the amounts of their speckle shifts were

proportional to k, the renal pixel displacement

coefficient, and other equations and coefficients

that were analyzed in the previous sections. The

amounts of the speckle shifts were tuned from 0

to 15 pixels, which were in correlation to the

amounts of thermal changes during the vascular

occlusion and revascularization (from 0C to

20C). The model presumes that the system form-

ing the image has a linear and space invariant

point spread function (PSF). If the biological tissue

is modeled by a continuous distribution of pointscatterers, whose impulse response is T(x,y), then

the radio frequency image I(x,y) can be obtained

by spatial convolution of them:

(8) I(x, y) = PSF(x, y) T(x, y).

By performing a spatial Hilbert transformation,

the final image was obtained as the envelope of

the radio frequency image I(x, y). In particular,

the PSF has been defined in terms of the acoustic

wavelength, , and the axial and lateral resolu-

tion of the imaging system by

(9)

We inserted a transducer frequency of 4 MHz,

sound velocity of 1540 m/s, wavelength of

0.385 mm, spatial frequency of sound of 5.195

cycles/mm, SD along the x- and y-axes of

0.426 mm, SD along the z-axis of 0.213 mm, x-

and y-axis range of 5 to 5 mm, and z-axis range

0 to 5 mm. The pattern was added to a mobile cir-

cular or multichamber cardiac model. A detailed

description of the B-mode ultrasound modeling

was published previously.24

Thermoacoustic Lens Effect

The ultrasound beam traversing the heated

region deviates as the thermal change alters its

velocity. The steep temperature gradient can

worsen this effect, which is usually negligible.

This phenomenon results in a regional ripple dis-

tal to the heated region. There are software and

hardware methods to reduce it; the former,

which is in our interest, is 2D spatial filtering. A

time-varying 2D gaussian filter as described

below will be used for postprocessing the ther-

mal change data:

(10)

where k= K/C, and n is the order of the filter.8

Converting Speckle Displacements to Thermal

Changes

Because the motion of each speckle is an accu-

mulated motion, the vertical motion vector is

derived once. Moreover, because derivationmakes the algorithm noise sensitive, we imple-

mented an order statistic filter of size 20, as

described previously.31,32According to Equations

1 and 2, displacements are converted to a thermal

shift. Finally, a gaussian kernel size of 10 pixels

and variance of 0.5 can smooth the result.

Motion Artifacts in the z-Axis

As we have discussed previously, optical flow has

been able to recover exact motion fields in the 2D

1540 J Ultrasound Med 2009; 28:15351547


Figure 3. Different motions of renal tissue and their extraction.


7/13

images even in the presence of large motion arti-

facts in the z-axis (such as cardiac motion detec-

tion in B-mode echocardiographic series).

The motion in the z direction is called an out-

of-plane error. There are 2 techniques to cope with the out-of-plane error: (1) temporal win-

dowing (spatiotemporal windowing) and (2)

confidence measurement.

1. Temporal windowing (spatiotemporal win-

dowing). This technique applies a gaussian kernel

among the consecutive frames (using a weighted

summation of the consecutive frames). This strat-

egy reduces the effect of the out-of-plane error,

especially when the temporal kernel is large.

Moreover, temporal windowing dramatically

decreases the power of noise and increases the sig-

nal to noise ratio, which is a helpful phenomenonin noise-sensitive optical flow techniques.

2. Confidence measurement. The robustness

and sensitivity of the optical flow methods are

not the same among different regions of an

image. Considering a confidence measurement

for the motion estimation increases the accura-

cy, especially for the out-of-plane error regions.

A detailed discussion of confidence measure-

ment was published previously.30

Clinical Evaluation

Any medical system should be implemented on

real tissue before practical application. We selected

3 healthy sheep and then occluded a renal seg-

mental artery in each to observe the heat changes

in that special lobule (each lobule is supplied by

a segmental artery). Theoretically, we should have

had a decrease in the renal lobular temperature

after occluding the segmental artery. The occlusion

was done surgically, and a gradual reopening of the

segmental arterial branch was performed in nearly

2 hours to model anticoagulation reperfusion med-

ications ideally. The animals were anesthetizedwith midazolam and morphine. The incision was

done as a great midline incision. The whole surgery

lasted 1 hour and was finalized with a bolus of

potassium chloride injected intravenously.

Results

B-Mode Sonographic Simulation

The coefficients described in Renal B-Mode

Sonographic Simulation above were used in the

sonographic simulation. Finally a 20-frame

sonographic sequence was obtained, in which 1

point was considered the hot point. The temper-

ature of the hot point varied from 0C to 50C.

The appropriate thermal coefficient and motionartifacts were assigned to the simulation.

Different motion estimation techniques were

implemented and optimized using different

parameters. In this study, the evaluation was per-

formed on simulated images to find out the best

algorithm and optimized coefficients.

Evaluation of Motion Estimation Methods on

the Simulated Data (Finding the Optimized

Method)

According to the algorithm, which was discussed pre-

viously, we synthesized the computerized B-modesonographic movies. The speckle-tracking tech-

niques were evaluated on these simulated images.

All sequences were prefiltered using a gaussian

kernel of size 20 pixels and variance 0.5. For the

Lucas-Kanade method, we experimentally found

that a local window of size of 5 pixels was the best

one. All other parameters were set as described

by Barron et al.35 For the Horn-Schunck method,

an iteration number equal to 300 was found to

be the best. The number was optimized

between 0 to 1000, and 600 was found to be the

best. Gaussian spatial and temporal windows

were considered 15 pixels and 10 frames, respec-

tively. All the applications were performed in

MATLAB 7.1 (The MathWorks, Natick, MA) with a

3-GHz processor (Intel Corporation, Santa Clara,

CA), and 1028 MHz of RAM.

It is a very popular evaluation to implement the

motion estimation techniques on the simulated

sequences.35 Errors were computed with the

widely used amplitude (Equation 11) and angu-

lar errors (Equation 12):

(11)

(12)

where V, Vdefine the real and estimated vectors.33

J Ultrasound Med 2009; 28:15351547 1541

Abolhassani et al


8/13

Figures 47 show the results of the implementa-

tion of different motion estimation techniques on

the simulated images with respect to the average

amplitude and angular error. The gradient-based

optical flow methods show better results than theblock-matching methods except the ES method.

According to the images, newer block-matching

techniques have better performance because of

their adaptive behavior with respect to the previ-

ous frame and their better searching strategy in

the diamond search and 3SS. The performance of

older block-matching techniques such as the 3SS

is lower than that of the newer ones. The exhaus-

tive and correlation-based methods are accurate

but more computationally expensive. The corre-

lation algorithm is the most time-consuming

technique, and implementing the gradientdescent cannot dramatically remove this prob-

lem. There was no significant difference between

the MAD and MSE, but the MSE takes more time.

Among the optical flow techniques, the moment-

based approaches act better because of their

invariant hypergeometry. Horn-Schunck outper-

forms the popular Lucas-Kanade due to its higher

robustness to the out-of-plane error. The imple-mentation of the phase matching has a small

effect on increasing the accuracy. Increasing the

optical flow iterations can increase the accuracy

but also increases the time consumption.

Table 1 shows the results of implementation of

the optimized optical flow (multiresolution Horn-

Schunck) and optimized block-matching (ARPS)

techniques for thermal detection. To determine

the thermal changes, motion vectors are derived

once with respect to the vertical axis. Therefore,

the robustness to noise and achieving smooth

results will be very important. Theoretically, theHorn-Schunck energy function has a smoothing

term that achieves smooth and visually attrac-

tive results. Therefore, thermal detection is more

accurate based on the Horn-Schunck algorithm.

Moreover, the multiresolution moment imple-

mentation of the Horn-Schunck technique has

the ability to achieve invariant features and

reduce noise (contrary to their high-pass nature).

Figure 8 shows 2 frames of the synthetic

sequence before and after the thermal shift. Figure

9 is the result of the implementation of the most

accurate algorithm (Horn-Schunck) using the

optimized coefficients on the mentioned frames.

The thermal motion is extracted after subtracting

the interpolated motion from the total motion.

The left image is the total motion vector and is pre-

dominantly made up from the motion artifact.

The right image shows the net thermal shift after

motion artifact removal using the proposed

method.

Figure 10 is the color-coded result of the ther-

mal change in the same frames. The top image

shows the real thermal change (ground truth), which was used in the model, and the bottom

images show the result of the optimized method

in the presence of a motion artifact (left image)

and without a motion artifact (right image).

In Vivo Results Based on the Optimized Method

So far, it has been shown that the multiresolu-

tion Horn-Schunck method is the optimized

technique for simulated ultrasound images.

We implemented this algorithm on the real data

1542 J Ultrasound Med 2009; 28:15351547


Figure 4. Comparison of different block-matching speckle-tracking methods usedon the simulated data.



9/13

during occlusion and reopening of the renal

artery, only the optimized motion estimation

technique, ie, multiresolution Horn-Schunck,

was applied to the sequences, and the coeffi-

cients were set the same as for the simulatedsequence validation. It is very difficult to quan-

titatively analyze the thermal change on real

animal frames, and we validated the in vivo

results on the basis of the clinical knowledge of

physicians.

The real sequences were obtained using a

probe frequency of 7 MHz and a temporal reso-

lution of 25 frames per second. Three sheep were

anesthetized successfully using midazolam and

morphine. All of the subjects remained alive dur-

ing the whole surgery, and no significant operat-

ing room side effect happened.Figure 11 shows 2 frames of the sequence,

which were taken 0 and 10 minutes after the

reopening. The occluded artery in not visible

in the ultrasound images, but the thermal

change results of the arterial occlusion can be

detected.

1. In the first step, the total motion is extracted

(combined motion artifact and thermal change)

for the ROI and ROI neighborhood using the

optimized method. The ROI (a 200 200 box) is

defined manually for the first frame. The ROI

neighborhood pixels are defined as 100 pixels

around the primary ROI. Figure 12 shows imple-

mentation of the optimized algorithm (mul-

tiresolution Horn-Schunck) on the real

ultrasound images in the ROI and ROI neighbor-

hood pixels for the mentioned frames.

2. In the next step, the interpolated motion vec-

tors of the ROI are computed on the basis of the

motion vectors of the ROI neighborhood pixels.

We implemented cubic Ispline interpolation in

2D followed by mild gaussian smoothing (vari-

ance, 0.1; size, 5 5). Figure 13, left image, showsthe total motion vector for the mentioned frames.

3. To extract the pure thermal change, we sub-

tracted the total ROI motion vectors (Figure 13, left

image) from ROI motion artifact (Figure 13, cen-

ter image). Figure 13, right image, shows the total

motion vector in the ROI region.

4. The vertical thermal motion vectors are

derived with respect to the vertical axis to deter-

mine the net thermal change. Figure 14 shows

the thermal elevation in the ROI.

As proved, the lower renal lobe temperature

increases. It is evident that there is a warm-up

gradient in the temperature in the bottom of the

renal tissue, which correlates well with the

anatomy of the lower renal lobe perfusion. With

respect to the fact that the lower renal segment

was being occluded and reopened in that case,

the lower renal part warm-up is noteworthy after

the reperfusion.

Figure 15 shows the thermal change between 0

and 5, 0 and 10, 0 and 15, and finally 0 and 20minutes in the sonographic series. In the last 2

J Ultrasound Med 2009; 28:15351547 1543

Abolhassani et al

Figure 6. Results of implementation of optical flow techniques on the simulateddata.


Table 1. Thermal Detection Results of the Best Motion EstimationMethod of Each Category

Thermal Change Average Variance of ThermalMethod Error, C Change Error, C

ARPS 1.2 2.4Horn-Schunck 1.02 1.9Multiresolution Horn-Schunck 0.50 0.502


10/13

color images, even branching of the central lobar

artery is substantial (although sonography can-

not even show the main artery). Note that ther-

mal change estimation cannot extract the

primary temperature but only the thermal

change between 2 frames.

We averaged the temperature change of the ROI

of each frame. Figure 16 shows that the averagedtemperature in the ROI increases after reopening

of the arterial segment (warming section). The

results of the real data were validated by clinical

experts. The plots in Figures 15 and 16 are well

related to the clinical knowledge of the lower seg-

mental renal artery.

Discussion

We have proposed a novel real-time and nonin-

vasive method for thermal detection based on

ultrasound velocity changes in tissue matter.

This technique was used to follow revasculariza-

tion therapy in renal tissue after renal arterial

occlusion. The motion artifacts were carefully

removed. Implementation of the algorithm on

the real and the simulated data showed a good

correlation among them.

1544 J Ultrasound Med 2009; 28:15351547


Figure 8. Simulated thermal change in renal tissue. The boxesshow the thermal detection ROI.

Figure 11. Two frames of the renal sonographic series immedi-ately and 10 minutes after reopening. Note that the renal lower

lobe artery cannot be seen. The boxes show the ROI, and the

line indicates a slight vertical shift.

Figure 9. Left image, Motion vectors of the abdominal sono-graphic model in the ROI of the aforementioned frames. Right

image, Thermal change vectors after subtracting the motion arti-

fact using the optimized method (multiresolution Horn-Schunck).

Figure 10. Top image, Thermal change in the renal tissue.Bottom, right image, Result of implementation of the multireso-

lution Horn-Schunck technique (the most accurate method)

using the optimized coefficients in the presence of a motion arti-

fact. Bottom, left image, Same implementation without a motion

artifact. Obviously, the motion artifact decreases accuracy.

Figure 12. Motion vectors of the ROI and ROI neighborhood pixels,which are a summation of thermal changes and a motion artifact.


11/13

Among all of the motion estimation techniques

analyzed, the ARPS was the best block-matching

method, whereas the moment-based multireso-

lution Horn-Schunck technique was the best

gradient-based method. However, the latter out-

performed the former; therefore, we used the

multiresolution Horn-Schunck technique for the

thermal change extraction.The ARPS was the best block-matching method

because of its adaptive strategy, which adjusts

the next motion vectors according to the previ-

ous vectors. The multiresolution approaches

were able to extract different values of thermal

change, overcome the local minima problem,

and achieve more reliable outputs. Fine motions

were preserved using finer moments, and large

and coarse motions were preserved. For the sim-

ulated images, the average thermal error was

0.5C with an SD of 0.5C, although much lower

errors can be obtained in noiseless (motionless)

data. This method can be an efficient technique

for revascularization therapy evaluation and

follow-up based on digital B-mode sonographic

methods. This study was also a good evaluation

of several motion estimation techniques for

speckle tracking.

We are currently working on multigrid meth-

ods to achieve faster techniques. Moreover, we

are interested in implementing this algorithm in

other clinical problems. Many vital changes in

the human body also change the tissue temper-

ature; therefore, thermal detection can be a ref-

erence standard for tracking them. The most

attractive but complex one is detecting the

myocardial temperature during streptokinase-

based coronary revascularization, which is very

difficult because of the great myocardial contrac-

tion and relaxation.

J Ultrasound Med 2009; 28:15351547 1545

Abolhassani et al

Figure 13. Left image, Total motion vectors of the ROI. Centerimage, Interpolated ROI motion vectors using ROI neighborhood

pixels. Right image, Subtraction of the left image from the cen-

ter images, which is the thermal change.

Figure 15. Renal B-mode sonograms and their extracted thermal changes afterarterial reopening in the ROI. The first image (a) was taken 10 minutes after the

revascularization therapy, whereas the others were taken 20 (b), 30 (c), and 40 (d)minutes after the revascularization therapy respectively. There is a gradual ROIwarm-up when the artery is opened, and even branching of the vessel is evident

in the last frame. e, Close-up of the thermal change in d.

Figure 14. Thermal change after derivation of accumulatedthermal displacements in Figure 13, right image. The color bar

shows the thermal change in degrees Celsius.

Figure 16. Renal lower segment thermal elevation after reperfusion.The x-axis is the time in minutes, and the y-axis is the temperature

of the tissue. Notice the close correlation with the clinical situation.

Author: In Figure 14 legend, please verify Figure 13, right

image, as changed from Figure 9c.


12/13

References

1. Kasper DL, Braunwald E, Fauci AS, Hauser S, Longo D,

Jameson JL. Principles of Internal Medicine. 16th ed. New

York, NY: McGraw-Hill; 2005.

2. Abolhassani MD, Norouzi A, Takavar A, Ghanaati H.

Noninvasive temperature estimation using sonographic

digital images. J Ultrasound Med 2007; 26:215222.

3. Gertner MR, Worthington AE, Wilson BC, Sherar MD.

Ultrasound imaging of thermal therapy in in vitro liver.

Ultrasound Med Biol 1998; 24:10231032.

4. Adam D, Raukhverger R. Real-time monitoring hyperther-

mia therapy by ultrasound imaging. Paper presented at:

Mediterranean Conference on Medical and Biological

Engineering; July 31August 5, 2004; Ischia, Italy.

5. Ueno S, Hashimoto M, Fukukita H, Yano T. Ultrasound

thermometry in hyperthermia. IEEE Ultrason Symp Proc

1990; 3:16451652.

6. Varghese T, Daniels MJ. Real-time calibration of tempera-

ture estimates during radiofrequency ablation. Ultrason

Imaging 2004; 26:185200.

7. Varghese T, Zagzebski JA, Chen Q, et al. Ultrasound moni-

toring of temperature changes during radio frequency

ablation: preliminary in-vivo results. Ultrasound Med Biol

2002; 28:321329.

8. Simon C, Vanbaren P, Ebbini ES. Two-dimensional temper-

ature estimation using diagnostic ultrasound. IEEE Trans

Ultrason Ferroelectr Freq Control 1998; 45:10881099.

9. Seip R, Vanbaren P, Cain A. Noninvasive real-time multi-

point temperature control for ultrasound phased array

treatments. IEEE Trans Ultrason Ferroelectr Freq Control1996; 43:10631073.

10. Straube WL, Arthur RM. Theoretical estimation of the tem-

perature dependence of backscattered ultrasonic power

for noninvasive thermometry. Ultrasound Med Biol 1994;

20:915922.

11. Karjalainen T, Thierman JS, Hynynen K. Ultrasound acous-

tic stimulated emission for controlling thermal surgery. IEEE

Ultrason Symp Proc 1999; 2:13971400.

12. Konofagou E, Thierman J, Hynynen K. The use of ultra-

sound-stimulated acoustic emission in the monitoring of

modulus changes with temperature. Ultrasonics 2003;

41:337345.

13. Konofagou E, Thierman J, Karjalainen T, Hynynen K. The

temperature dependence of ultrasound-stimulated acous-

tic emission. Ultrasound Med Biol 2002; 28:331338.

14. Arthur RM, Straube WL, Trobaugh JW, Moros EG. Non-

invasive estimation of hyperthermia temperatures with

ultrasound. Int J Hyperthermia 2005; 21:589600.

15. Furht B, Greenberg J, Westwater R . Motion Estimation

Algorithms for Video Compression. Waltham, MA: Kluwer

Academic Publishers; 1997.

16. Li R, Zeng B, Liou ML. A new three-step search algorithm

for block motion estimation. IEEE Trans Circuits Syst Video

Technol 1994; 4:438442.

17. Lu J, Liou ML. A simple and efficient search algorithm for

block-matching motion estimation, IEEE Trans Circuits Syst

Video Technol 1997; 2:429433.

18. Po L, Ma WC. A novel four-step search algorithm for fast

block motion estimation, IEEE Trans Circuits Syst Video

Technol 1996; 6:313317.

19. Zhu S, Ma KK. A new diamond search algorithm for fast

block-matching motion estimation. IEEE Trans Image Process

2000; 9:287290.

20. Nie Y, Ma KK. Adaptive rood pattern search for fast block-

matching motion estimation. IEEE Trans Image Process

2002; 11:14421448.

21. Cheung C, Po L. A novel small cross-diamond search algo-

rithm for fast video coding and video conferencing appli-

cations. In: Proceedings of the IEEE International

Conference on Image Processing. Piscataway, NJ: Institute

of Electrical and Electronics Engineers; 2002.

22. Cheung C, Po L. A novel cross-diamond search algorithmfor fast block motion estimation. IEEE Trans Circuits Syst

Video Technol 2002; 12:11681177.

23. Horn BKP, Schunck BG. Determining optical flow. Artif Intel

1981; 17:185283.

24. Baraldi P, Sarti A, Lamberti C, Prandini A, Sgallari F,

Evaluation of differential optical flow techniques on syn-

thesized echo images. IEEE Trans Biomed Eng 1996;

43:259272.

25. Lucas BD, Kanade T. An iterative image registration tech-

nique with an application to stereo vision. In: Proceedings

of the International Joint Conference on Artificial

Intelligence. San Francisco, CA: International Joint

Conferences on Artificial Intelligence; 1981:674679.26. Sahba N, Tavakoli V, Ahmadian A, Abolhassani MD,

Fotouhi M. Hybrid local/global optical flow and spline

multi-resolution analysis of myocardial motion in B-mode

echocardiography images. Tech Acoust 2008. http://

www.ejta.org.

27. Tavakoli V, Sahba N, Ahmadian A, Amini AA. Adaptive

Multi-resolution myocardial motion estimation in B-mode

echocardiography images. In: Proceedings of the IEEE

International Conference on Bioinformatics and Biomedical

Engineering. Piscataway, NJ: Institute of Electrical and

Electronics Engineers; 2008:20332036.

28. Tavakoli V, Sahba N, et al. An evaluation of different optical

flow techniques for myocardial motion analysis. In:

Proceedings of the IEEE International Conference on

Bioinformatics and Biomedical Engineering. Piscataway, NJ:

Institute of Electrical and Electronics Engineers; 2008:2528.

29. Tavakoli V, Sahba N. A new variational technique for com-

bining affine registration and optical flow in echocardiog-

raphy images. In: Proceedings of the 30th Annual

International IEEE Engineering in Medicine and Biology

Society Conference. Piscataway, NJ: Institute of Electrical

and Electronics Engineers; 2008:205208.

30. Shling M, Arigovindan M, Jansen C, Hunziker P, Unser M,

Myocardial motion analysis from B-mode echocardio-

grams, IEEE Trans Image Process 2005; 14:525536.

1546 J Ultrasound Med 2009; 28:15351547

Author: Please verify volume numbers in reference 5, 11, and 1618. Please give page

numbers for reference 21. Please verify reference 23; unable to find in our sources.


13/13

31. Mehrabani BM, Tavakoli V, Abolhassani MD, et al. An effi-

cient temperature estimation using optical-flow in ultra-

sound B-mode digital images, In: Proceedings of the 30th

Annual International IEEE Engineering in Medicine and

Biology Society Conference. Piscataway, NJ: Institute of

Electrical and Electronics Engineers; 2008:8689.

32. Sahba N, Tavakoli V, Nambakhsh MS. Ultrasound thermal

change detection based on steerable filters. In:

Proceedings of the 30th Annual International IEEE

Engineering in Medicine and Biology Society Conference.

Piscataway, NJ: Institute of Electrical and Electronics

Engineers; 2008:29842987.

33. Tavakoli V, Sahba N, Hajebi N, et al. A fast and accurate

method for echocardiography strain rate imaging. In:

Proceedings of the Annual Meeting of the International

Society for Optical Engineering. Bellingham, WA:

International Society for Optical Engineering; 2009:7259

72592.

34. Tavakoli V, Sahba N, et al. Two-dimensional echocardiogra-

phy motion analysis based on optical flow techniques.

Biomed Imaging Interv J. In press.

35. Barron JL, Fleet DJ, Beauchemin S, Performance of optical

flow techniques. Int J Comput Vis 1994; 12:4377.

J Ultrasound Med 2009; 28:15351547 1547

Abolhassani et al

Author: Please give all authors last names and initials for references 28,

31, and 33. Please give correct page numbers for reference 33. Also,

please give all authors last names and initials for reference 34 and

update all information if now available.

optimized noninvasive monitoring of thermal changes on digital b-mode renal sonography during...

Documents