virtual bronchoscopy

Virtual BronchoscopyG. McLennan1,2,3, E. Namati1, J. Ganatra4, M. Suter5, E. E. O’Brien1, K. Lecamwasam1, E. J. R. van Beek1,2,E. A. Hoffman1,2,3

1Department of Internal Medicine, University of Iowa, Iowa City, IA, USA2Department of Radiology, University of Iowa, Iowa City, IA, USA3Department of Biomedical Engineering, University of Iowa, Iowa City, IA, USA4VIDA Diagnostics Inc., Iowa City, IA, USA5Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, MA, USA

Correspondence to:Geoffrey McLennan, M.D., Ph.D.Departments of Internal Medicine, Radiology, and Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USATel: 319 356 3603; Fax: 319 353 6406; E-mail: [email protected]

Key words: Bronchoscopy, macro-optical, micro-optical, eidomics, personalized medicine, computed tomography.

Summary

Virtual bronchoscopy is evolving rapidly, and is

becoming accepted into standard clinical practice. Vir-

tual bronchoscopy is a term, which encompasses not

only multi-row detector X-ray computerized tomogra-

phy-derived images, but also other computer graphics

and computer vision-derived value-added digital imagery.

Other imaging data sources used to create three-

dimensional image renderings of the bronchial tree

include magnetic resonance imaging, positron emission

tomography, the digital colour image taken at real

bronchoscopy as part of macro-optical imaging, and

various emerging micro-optical imaging modalities, such

as catheter-based confocal microscopy and optical

coherence tomography. Software solutions now exist for

providing simple renderings of the bronchial tree

through which a fly-through of the airway lumen along

the centreline of the airway can be added (the fly-

through mimics the view that a real flexible broncho-

scope affords the operator). The images so produced are

visually accurate and with currently available software

also analytically correct. More advanced virtual

bronchoscopic applications, including image-based path-

finding to mediastinal and peripheral lung structures, are

also in development, and are finding their way into

clinical studies. Exciting and synergistic data sets com-

posed of image data from multiple image sources are

also being constructed. One emerging issue is to enhance

the understanding and reporting on these data sets,

which are often complex, and which are full of useful as

well as redundant information. The new discipline of

eidomics will inform the non-specialist end user, and act

to predict important outcomes. These increasingly

powerful tools will continue to advance the use of ima-

ging in technology-supported personalized medicine, to

compliment the information from genomics.

Introduction

The bronchial tree is the primary site of common diseases

including lung cancer, chronic obstructive pulmonary

disease (COPD) and asthma, yet the assessment of the

bronchial tree has until recently been restricted in selected

subjects to non-analytical subjective light-based bronchos-

copy, through either a rigid or flexible tube, or even less

discriminatory pulmonary function tests. Virtual bronch-

oscopy initially was the descriptive term given to repre-

sentations of the bronchial tree and surrounding structures

created from spatial information derived from imaging

sources other than the bronchoscope itself. The term was

applied to the two-dimensional and later three-dimensional

representations of the bronchial tree obtained from multi-

row detector X-ray computerized tomography (MDCT)

(1), although now the term equally and increasingly applies

to similar images derived from magnetic resonance ima-

ging (MRI), and from positron emission tomography (PET)

imaging (2, 3). Further, macro-optical traditional bronch-

oscopy images are now increasingly digital, allowing im-

age-processing techniques to be applied to the standard

two-dimensional modern bronchoscope image, providing

another source of virtual bronchoscopy images (4, 5). As

digital-based micro-optical techniques are developed, these

also can be applied to the bronchial tree, increasing both

the visualization of the bronchial tree, and its micro-

structural components. Image processing techniques are

being used much more frequently, and out of necessity, on

these image sets for both image display and analysis. Thus,

the toolbox for virtual bronchoscopy has greatly expanded.

MDCT of the lung produces two-dimensional images

(a cross-section of the thorax at the slice point) with the

minimal x, y resolution in this image referred to as a ‘pixel’

and the depth of the slice adding a z direction to that pixel;

this volumetric minimal resolution image is referred to as a

‘voxel’. Over the last 10–15 years, the x, y and z resolutions

IMAGING DECISIONS n 1/2007

of the CT scans have dramatically improved along with

improved scan acquisition times. Current CT scanners (64

multi-row detector CT devices) now produce isometric

voxels of the order of 0.4 mm. Stacks of CT slices,

adequately reconstructed, now allow for high-resolution

three-dimensional images of the thorax to be obtained.

This simple explanation also applies to most other digital

imaging systems. The three-dimensional image of the

thorax obtained through MDCT is a digital image, with

each voxel having a defined spatial dimension and grey-

scale characteristics. As with any digital data set, the three-

dimensional image of the thorax can be analysed, digitally

stored and retrieved, and displayed as an image-based

informational structure. The three-dimensional image

contains all of the structures within the thorax, including

the airways (where there is natural contrast between tissue

and air), the mediastinal blood vessels (where, with contrast

injection, there is discrimination between blood vessels and

other soft tissue structures), and the mediastinal lymph

nodes (where there is often significant contrast differences

between the lymph node and surrounding mediastinal fat).

Similar structures within the three-dimensional image data

set can be digitally removed from the remainder of the

image for later visualization and analysis.

Using these principles of digital technology it is reason-

able to expect that the bronchial tree (and the surrounding

structures if needed) can be identified, then removed from

the larger image structure and evaluated in three-dimen-

sional digital space (Fig. 1). This is the basis of MDCT-

based virtual bronchoscopy and similar to the virtual

bronchoscopy views derived from other digital image

sources. Until recently, clinical software solutions could not

keep up with the rate at which scanner hardware

technology was developing (6, 7). The rapid increase in

the amount of data in MDCT image sets mandates that

computer-aided interrogation of the images occurs to assist

the human reader before a final report is issued. Such

computer-aided assistance is increasingly available to not

only display selected anatomical image sites, such as the

bronchial tree, but also for a measurement task. Implicit in

this advance is the need for standardization of image

acquisition devices, as well as uniform calibration approa-

ches. Such standardization processes are now being

discussed (8).

Simple MDCT-based virtual bronchoscopy

Since the mid 1990s, when MDCT-based virtual bronch-

oscopy was first introduced into clinical practice, significant

effort and resources have been put towards automatically

segmenting the airway tree and lung from the CT-obtained

images in academic research laboratories (9) and in

industry. Most of the major CT manufacturers now offer

MDCT-based virtual bronchoscopy software, and there

are several smaller companies offering MDCT-based vir-

tual bronchoscopy software. For software packages to be

useful, three conditions must be satisfied. First, they must

provide rapid and adequate visualization of the three-

dimensional airway structures and accurately depict the

relationships of the bronchial tree with surrounding

structures, such as major blood vessels. Second, the visu-

alized results must not only be anatomically correct but

also provide accurate measurements of the airway lumen

for length and diameter of suspect segments – examples are

shown in Fig. 2. This is extremely important for inter-

ventional pulmonology applications, such as bronchial

stent placement. Finally, for the software to be clinically

accepted and used, it must be available in a usable format

at the point of service for the medical care or procedure

being performed.

The accuracy of MDCT-based virtual bronchoscopy

techniques with real bronchoscopy findings is high (10–12),

j Fig. 1. Showing the MDCT-based bronchial tree isola-ted from the full MDCT scan of the chest, in the large image,and taking several seconds to automatically segment out theairways, lungs and lobes. The lungs have also beenisolated, and are displayed in conjunction with the bronchialtree, although given a different colour, with each lobe clearlydefined and presented as well. The relationships betweenthe lungs, lobes and airways are precise and anatomicallycorrect. Note that the bronchial tree is displayed a long wayout into the periphery – further than can be viewed with atraditional flexible bronchoscope. If needed this image canbe rotated, and viewed from any angle. A fly-through of thebronchial tree can also be performed. In the top left handcorner is an MDCT-based bronchial tree segmented out byhand 10 years ago. This process took several hours. Notethe pixellations from the relatively thick slice MDCT scanperformed as state of the art at that time. Additionally, onlythe central airways could be segmented, due to lack ofimage resolution. The smaller image shows a stenosed rightmain bronchus post lung transplant. Large image developedusing software from VIDA Diagnostics Inc.

V I R T U A L B R O N C H O S C O P Y n 1 1

1/2007 n IMAGING DECISIONS

and this will improve further as CT scanning protocols

improve. This is particularly useful for those conditions

where there may be multiple stenoses, such as Wegener’s

granulomatosis (13), or secondary cancers where the

traditional bronchoscope cannot visualize past the first

obstructing segment. MDCT-based virtual bronchoscopy

techniques in the follow-up of patients with known

bronchial stenosis, such as right main bronchus stenosis

at the anastomosis site after lung transplantation where,

after an initial procedure, such as balloon dilatation, the

stenosis can be followed over time using virtual bronch-

oscopic techniques without putting the patient through a

further real bronchoscopic evaluation. There is an emer-

ging consensus that MDCT-based virtual bronchoscopy is

a useful adjunct in clinical management of a variety of

tracheobronchial conditions, as described in a recent meta-

analysis (14). Furthermore, MDCT-based virtual bronch-

oscopy techniques provide useful information about the

relationship of any bronchial abnormality of surrounding

structures, such as the manubrium sterni in a high tracheal

stenosis where there may be a question as to whether

tracheostomy can be safely performed. Finally, these

virtual bronchoscopic procedures help with bronchial stent

sizing before the procedure where one can plan both the

length and diameter of the stent, or for an initial

assessment of balloon sizes for balloon dilation within the

airway (15, 16). This preplanning ensures that the required

balloon or stent is in the inventory and that the inserted

device fits the airway appropriately, leading to less stent

migration and less granulation tissue at the ends of the

stent. It seems clear that the MDCT-based virtual

bronchoscopic view of the airway, if both anatomically

accurate and providing accurate measurements, is an

important new addition in the pulmonary physician’s

practice, and is especially useful in evaluating the airway

before initial or sequential real bronchoscopies where an

interventional procedure might be considered. These

images reduce the chances of significant surprises occur-

ring during any procedure, and allow the patient and

family to be as fully informed as the physician of record.

These descriptions are that of simple MDCT-based

virtual bronchoscopy techniques and applications. This

increasingly informs the current standard practice especi-

ally for interventional pulmonology procedures where

preplanning of complex procedures is essential in improv-

ing operative and postoperative outcomes.

j Fig. 2. This is a screen shot of a human virtual bronchoscopic image set showing the value of complex but fast computer-based segmentation and image analysis techniques. The primary data set is an MDCT scan set. The blue coloured airway inthe centre lower panel represents a pathway chosen for measurements, from the trachea down into the left main bronchusand into the left lower lobe. The computer program then straightens out the chosen airway pathway and displays this in thetop panel. The airway can then be measured at any plane for any dimension, including minor and major diameters, or surfacearea. Additionally wall thickness is calculated at that point. The original MDCT data are displayed at the lower left panel forany particular chosen point on the straightened airway, so relationships with structures outside of the airway can be seen incontext. The virtual bronchoscopic fly-through view at the nominated plane is also displayed on the right-sided lower panel.The concentric rings dotted around the inner lumen of the virtual fly-through represent the measurement planes from theupper image. Each of the panels displayed are linked in real time – if the plane of measurement is moved then all of theimages adjust instantly. Images developed using software from VIDA Diagnostics Inc., Pulmonary Workstation 1.2.

1 2 n V I R T U A L B R O N C H O S C O P Y


Complex MDCT-based virtual bronchoscopy, with

pulmonary pathfinding

Complex virtual bronchoscopy procedures are now mov-

ing from the research laboratory into clinical studies and

some of them will be in clinical practice over the next

several years.

The first of these complex virtual bronchoscopy proce-

dures examined here uses the data contained within the

complex MDCT-based three-dimensional image for pro-

cedure guidance within the mediastinum and hilar struc-

tures (17, 18). A simple example is in understanding where

a mediastinal lymph node is in relationship to the

bronchial tree. In traditional bronchoscopy, only the

airway lumen is visualized, although the bronchoscopist

knows that somewhere on the other side of that non-

transparent bronchial epithelium is the lymph node that is

the target for a transbronchial needle aspiration or core

biopsy. Using the virtual bronchoscopy information

obtained from the three-dimensional MDCT data set, a

virtual bronchoscope (placing a visualization tool where

the real bronchoscope might be within the lumen) can

show precisely the same image as the real bronchoscope

does, with grey-scale rendering rather than the colour

rendering from the real bronchoscope image. The bronch-

oscopy operator can, with a great degree of confidence,

know where the target lymph node is. This process can be

simplified before the real bronchoscopic procedure in a

preplanning step where the target region, be it lymph node

or some other structure, can be rendered as a region of

interest and displayed as a colour object in both the virtual

bronchoscopic and later the real bronchoscopic images.

There are several computer programs that have been

developed for this application that are currently in phase

III clinical studies, with early results suggesting satisfactory

ease of use and improvement in the biopsy return. These

phase III clinical studies have used experienced bronchos-

copists as the operating gold standard, and one imagines

that the average bronchoscopist will have significantly

better yields with transbronchial needle procedures using

this type of image-guided computer assistance, again at the

point of service in the bronchoscopic laboratory. Early

results suggest a greater than 90% success rate for

mediastinal and hilar lymph node biopsies. Clearly, these

early successes open the possibility of sampling mediastinal

structures precisely and reliably. This also opens up the

possibility of localized precise application of non-specific or

specific therapies.

A second advanced virtual bronchoscopic application is

that of pathfinding to a peripheral region of interest within

the lung (19). Given the respiratory motion that occurs

during breathing, transcutaneous approaches to the mov-

ing lung may not be satisfactory. By approaching the

peripheral lung through the bronchial tree, movement is

not an issue because the parenchymal lung lesions move

synchronously with the airway and with any device that is

within the airway. With the significant improvement in

MDCT scanners, seven or eight generations of airways can

now be automatically extracted and evaluated. The simple

application here is that if the trachea is the beginning point

and if a pulmonary parenchymal abnormality (pulmonary

nodule) is the targeted end point, then appropriate

software can interrogate the three-dimensional image data

set and provide a pathway through the airway to the lesion

(Fig. 3). The bronchoscopist can simply follow this path-

way during a real bronchoscopy procedure and the correct

airway pathway to the lesion quickly cannulated using a

Teflon-coated tube that is very similar to the internal

coating of the standard bronchoscopic instrument channel.

Once the Teflon access tube is in place, then multiple

probes can be placed either to brush or biopsy, or optically

or by ultrasound sampling the lesion of interest. Ultrathin

bronchoscopes can be used in a similar manner (20–22).

Using these sorts of approaches, around 80% of peripheral

lung lesions can be easily and satisfactorily sampled. These

pulmonary pathfinding applications are in clinical studies

and are being developed by a number of companies for

medical application. They may have a synergistic role also

when coupled with magnetic tracking devices (23–25),

although the additional value of the magnetic tracking has

not yet been fully determined, especially in the peripheral

breathing lung. Nevertheless, the technology for magnetic

tracking is improving constantly, facilitated by MDCT

pathfinding, with recent useful clinical application (26).

The third advanced virtual bronchoscopy application

involves the targeting of the peripheral lung for endobron-

chial valve-type procedures in the management of pul-

monary emphysema, so-called endobronchial lung volume

reduction surgery (27–29). Here, the information that is

required is the state of the lung parenchyma and the extent

of emphysema in each segmental region together with the

anatomic configuration and size of the subtending airway

segments. Currently, several companies offer hardware

solutions for the endobronchial management of emphys-

ema and most of them are in human clinical studies. To

shorten procedure times, to improve accuracy of device

placement, to reduce medical error, and to educate the

patient and families, application of advanced virtual

bronchoscopy techniques is an obvious solution. For

instance, if the right upper lobe is being targeted (in some

studies, this is through human reading of the MDCT scan;

in others, it is by computer reading) it is unclear to the

operating interventional pulmonologist how many seg-

ments might require a device to be placed and whether the

segment lengths are adequate for the valve placement.

Such planning including valve sizing should be performed

where possible before the procedure by the management

team. In the immediate future, rather than targeting lobar

airways with therapeutic devices, it is likely that segmental

airway devices will be placed in those airways that subtend

areas of severe emphysema. In the lobar-only approach,

regions of the lobe that may not have much emphysema



may be treated unnecessarily. Segmental airway targeting

will likely require computer assistance and procedure

planning to improve accuracy of device placement. An

example of a software solution for valve and other

endobronchial device placement for the management of

emphysema and of pulmonary pathfinding is shown in

Figs 4 and 5. This software also automatically labels the

segmented airways to reduce operator confusion (30, 31).

Complex virtual bronchoscopy, with macro-optical

techniques

At the beginning of this article, it was indicated that virtual

bronchoscopy now applies to more than MDCT-based

data, but also to data from MRI, PET, and from the real

bronchoscopic images themselves. MRI- and PET-derived

data will not be discussed further except to make clear that

there is substantial work being performed in both of these

fields as they apply to virtual bronchoscopic applications,

with value-added computer vision and computer graphics

generated images for detection and characterization tasks.

In the remainder of this article we will discuss the virtual

bronchoscopic images obtained from regular two-dimen-

sional bronchoscopic images, together with the suggestion

that fusion of the multiple image data sets relating to the

bronchial tree (MDCT, MRI, PET and real broncho-

scopic) is likely to be synergistic in regard to providing

important detail about the state of a subject’s airways

particularly for early disease, such as lung cancer and

COPD.

The bronchoscopic images obtained through a real

bronchoscopic examination are now also digital images.

As such, there is an extraordinary amount of data. As

with digital radiology-type studies, much of the data

generated is not used by the clinical examiner. One

digital data set in modern bronchoscopy relates to the

colour of the airway mucosa. Here in the bronchoscopy

laboratory one often refers to ‘redder than normal’ or

‘pale mucosa’ without any analytic understanding as to

what that means and without any studies to indicate the

intraobserver or interobserver variability. This colour

information can now, through computer means, be

defined and recorded in a pixel-by-pixel basis and cross-

compared with other normative data or with disease

states (32–34). This advance has only been made possible

because of the digital nature of the bronchoscopes. It does

allow for comparative studies on airway mucosal colour

to be undertaken for the first time and for regions of

bronchial mucosa that lie outside of the normal range to

be highlighted for special consideration by the bronchos-

copist, such as possible biopsy or, in the future, some

form of optical sampling. Because colour and also texture

j Fig. 3. Computer screen shot of the pulmonary pathfinding application of virtual bronchoscopy. Here the bronchial tree isdisplayed from the MDCT data, together with the appropriate MDCT slice showing a peripheral pulmonary nodule in the leftpanel (selected and marked in yellow). Both the trachea and the centroid of the pulmonary nodule act as anchor points for theroad map (in blue) to the nodule that the bronchoscopist can follow during a flexible optical bronchoscopic procedure. Theright centre panel displays the virtual bronchoscopic endoluminal view, seen at the plane perpendicular to the airway at thelower portion of the trache on the left-sided image, with that portion of the MDCT slice shown immediately below. The selectedpathway is described verbally in the upper panel – trachea, right main bronchus, RB3, and further as needed. The wholesystem is dynamic and interactive with the operator at the point of service in the bronchoscopy laboratory. Images developedusing software from VIDA Diagnostics Inc.



assessment of the bronchial mucosa is so important in

bronchoscopic practice (e.g. this defines inflammation and

cancer) it seems reasonable that the future will see some

form of analytic colour bronchoscopy as a component in

the clinical workspace (Fig. 6).

The two-dimensional bronchoscopic image also contains

shadows attributed to the light source and its distribution,

both of which are known. Taking account of the shadow-

ing through a method known as ‘shape from shading’, a

three-dimensional bronchoscopic image can be computer

rendered from the two-dimensional images that most

bronchoscopists are used to viewing (Fig. 7). Because the

bronchoscopic colour information is digital, this can be

related to other digital imaging modalities, such as MDCT.

Indeed, the colour mucosal information from a regular

bronchoscopy can be painted on the virtual CT bronch-

oscopic images with a view to highlighting structures that

might be topographically abnormal (i.e. for shape) and

abnormal for colour. Computer software for these fused

applications has also been developed and is undergoing

further clinical evaluation.

If a functional image is fused, such as fluorescein

angiography taken at fluorescent bronchoscopy, then for

the first time one can evaluate the segmental airway with

multiple digital data sets providing a four-dimensional

virtual bronchoscopic evaluation (35). The three-dimen-

sional structural information combined with airway

mucosal changes over time from real-time bronchoscopy

provide the four-dimensional virtual bronchoscopy. This

has now been achieved; however, clinical use needs to be

defined.

Complex virtual bronchoscopy, with micro-optical

techniques

Utilization of pulmonary pathfinding techniques such as

outlined above from any of the sources of virtual bronch-

oscopy image data can provide road maps to almost any-

where that the bronchial tree supplies an airway to or near,

including as we have seen the mediastinum as well as the

peripheral lung. This allows for the development of pre-

cision-guided imaging techniques through the bronchial

tree – either ultrathin bronchoscopes, with traditional or

no small biopsy channels (36), or even smaller optical fibres

that take confocal, spectroscopy, or optical coherence

tomography samples. It remains unclear which techniques

might provide diagnostic optical images of suspect image-

based abnormalities, but within the bronchial tree confocal

j Fig. 4. Screen shots showing MDCT-aided virtual bronchoscopic applications in the bronchoscopic therapy foremphysema. The panel on the left shows two slices perpendicular to each other of the MDCT scan data (note not pixellatedas the voxels are isotropic). The emphysema in the lung parenchyma is well appreciated, as is the bronchial tree, displayedanatomically correctly in relation to the parenchyma. Note the bronchial tree carries an automatically labelled nomenclature toassist the operator, and that the lung lobes have been identified and different colours applied to them. On the right panel thecoloured balls now superimposed represent, on a lobar basis, the degree of emphysema, computer assessed throughcalculating low attenuation clusters (LACS), a measure of the bullae size in emphysema. Routine Hounsfield unit densityhistogram analysis can also be performed. These screens are also interactive with the viewer, and can be viewed frommultiple angles. The targeted segments for device placement can be identified, counted, and measured as a pre-planningstep for endo-bronchial device placement – this saves procedure time and allows the correct inventory to be carried on a just-in-time basis. Images developed using software from VIDA Diagnostic Inc., Pulmonary Workstation ‘‘Plus’’.



microscopy (37) and optical coherence tomography

(38–40) show promise. Even so the image data requires

some form of image processing to define abnormalities by

comparison to normative standards. So again there are com-

plex virtual bronchoscopic images at the microscopic level,

that can be integrated back into the macro level virtual

bronchoscopy renderings. An example of confocal micro-

scopy of the normal breathing lung is shown in Fig. 8 –

as can be seen the images can be viewed in 2D as well as in

3D, making the clinical bronchoscopists evaluation both

more comprehensive, as well as increasingly challenging.

Virtual bronchoscopy – immersive viewing

One problem with these complex information-rich 3D data

sets is to make them accessible and understandable to the

human observer. Issuing a written report is another major

problem, as we recognize that the written word is not

capable of describing an image, except in general terms.

Viewing a 3D image on a 2D page, such as in this article, is

not satisfactory, as much of the data is lost. Viewing the

information on a 2D computer screen, where the image

can be rotated by the viewer provides a much better

j Fig. 5. Lung segments displayed in the upper panels colour coded in each lung, on the left panel showing the view of thelungs from the front, centre panel from the side, right panel from the posterior aspect. The lower panel shows a single slice ofthe MDCT scan, with the segments again colour-coded, and evaluated by computer for the presence of, distribution of, andseverity of, pulmonary emphysema. Images developed using software from VIDA Diagnostics Inc.



j Fig. 6. The screen shot from a colour analysis computer program for macro-optical bronchoscopy. The colour wheel onthe left upper panel displays all colours, in hue and saturation. The grey region within the centre of the colour wheelrepresents normative bronchoscopic wall colour data from a training set. The black regions above the grey area are pixelsfrom the patient bronchoscopy images (shown in the right upper panel) that are outside of the normative data range. Thegreen pixels in the bronchoscopic image are the pixels considered by the computer to be abnormal in that image. The lowertwo graphs are the plot of hue and saturation for that image, with the blue curve showing the normative data, and the greencurves the image processed data. The panels on the right are statistical read-outs for that image. This program runs in realtime during a bronchoscopy, with each pixel in the image being assessed for variance from the normal range, producing avirtual bronchoscopic overlay map, on top of the real bronchoscopic image. These abnormal pixels can then be ‘painted’, ifneeded onto an MDCT-aided virtual bronchoscopy data set.

j Fig. 7. Another type of virtual bronchoscopic view, this time taken from macro-optical digital flexible bronchoscopicimages, on the left panel. Colour analysis is applied (but is not essential), showing a small mucosal cancer (with the greenpixels), not immediately obvious on the initial image, and then shape-from-shading algorithms run to generate a 3D virtualimage of the bronchial tree for those segments. As with other computer graphics images, this can be viewed from multipleangles, and measured for many parameters if needed.



viewing environment. Recently, virtual bronchoscopy

images have been ported to a fully immersive environment.

Under these conditions, there is a lot more detail seen and

appreciated by the human observer – see Fig. 9 for an

example. Desktop and shortly laptop viewing screens for

this immersive viewing experience are available to be used

at the point of service. The clinical utility of immersive

viewing of the complex data sets requires further study, but

it may of necessity become the standard in several years.

Eidomics

As can be seen the image data in these usually 3D data sets

contains an extraordinary amount of information. The

human observer has an unlimited ability to store and recall

images. Separating out the useful data as a detection task,

or as a characterization task, though, is increasingly diffi-

cult for the human observer, and master clinicians are

increasingly rare (41), in part because the wealth of infor-

mation invariably leads to a poverty of attention to detail,

and the clinical experience suffers. Much of the virtual

bronchoscopy image data is not visualized by the human

observer, although 3D viewing systems may improve that

significantly, as mentioned above. Having obtained the

best viewing technique, and utilizing computer systems to

assist with the detection and characterization of abnor-

malities, there is still the problem of utilization of the

images for education, especially of patients themselves, and

their non-specialist caregivers, and of using the image data

to focus on the few voxels that might constitute clinically

j Fig. 9. Immersive viewing of anMDCT-aided virtual bronchoscopyimages. The human observer canmanipulate the image with a smallcontroller, and can view the imagefrom any angle including a fly-throughif needed. This facility shown is aportal within the Center for Computer-Aided Design at the University of Iowa.

j Fig. 8. Confocal image of the peripheral alveolar region of the breathing lung, with a 2D slice from the image set on the leftpanel. The centre panel shows the 3D reconstruction of the alveoli in this region, excluding the connective tissue matrix. Theright panel shows the alveolar reconstructed images rotated and incorporated with the pleural lining just under the alveolarstructures. These digital micro-optical aided virtual bronchoscopy images are also measurable for size, volumes or areas, andcan be used to evaluate cell traffic in the peripheral lung. The images were obtained with a purpose-built research onlyconfocal system. Commercial systems are becoming available also through different vendors including Mauna KeaTechnology and Optiscan Ltd.



important abnormalities, with a written report that fully

describes the abnormalities. This can be achieved through

simulation and animation, where the focus is on the

abnormality, but with the addition of other known or as-

sumed pathophysiological structural and functional

behaviours. Additionally, a form of artificial intelligence

such as machine learning can then provide different sce-

narios that may predict the future course of that particular

disease process for that particular patient. This combina-

tion of functional computer-based features: complex 3D

images, simulation, animation and machine learning is

referred to as eidomics (42), and may well become part of a

new standard language of healthcare delivery in the future,

with a report issued as an image rather than as the written

word. A big advantage of this approach is that images are

recognized globally, making communication between dif-

ferent language speakers much easier, with less chance of

ambiguity.

Conclusions

Virtual bronchoscopy initially referred to the simple

depiction of the bronchial tree as a three-dimensional

image usually from MDCT data. This simple approach is

by itself not enough for useful clinical application; however,

if combined with accurate measurements in addition to the

visualization, then virtual bronchoscopy of this type has a

significant contribution to pulmonary practice particularly

for interventional pulmonology. This technology must be

available at the point of service. Advanced MDCT-aided

virtual bronchoscopy applications are now in clinical trials

including pulmonary pathfinding for peripheral lung le-

sions and mediastinal biopsies, preplanning for airway

procedures increasingly associated with emphysema man-

agement, and integration with novel verification systems

such as magnetic tracking. Utilization of the digital data

sets obtained through macro-optical flexible bronchoscopy

has lead to the development of computer-assisted evalua-

tion and display of the airway tree, providing a further level

of virtual bronchoscopy. Four-dimensional virtual

bronchoscopy has now been achieved in the laboratory

situation and might also shortly enter clinical studies. The

recent advent of catheter-based micro-optics such a con-

focal, optical coherence tomography or spectroscopic

techniques is providing new avenues for 3D image gen-

eration and analysis, resulting effectively in high-resolution

virtual bronchoscopy. Viewing these data sets, and merg-

ing them together for better human understanding is being

evaluated in several centres. Finally, the added complexity

of new knowledge generated through virtual bronchoscopy

applications, with multi-sourced information and with

multiple resolutions, can only be managed by the clinician

end user and the informed patient through the develop-

ment and application of eidomic techniques.

A clinical boundary is being crossed. In the past the

expert clinician end user viewed information gathered

from the patient in 2D, and made diagnostic and

therapeutic decisions based upon patient history, physical

examination findings, and supportive clinical tests, in the

context of clinical experience, with lumping of patients into

large diagnostic groups – such as COPD or a lung nodule

case. This made management relatively easy. In the future

the amount of information from 3D and 4D imaging will

be overwhelming to, and less interpretable by the clinician

end user – but will represent a data set that is entirely

unique to that patient. The patient history and physical

examination will remain fundamental, but very specific

information about the patient will be available – such as

COPD, but with 93% emphysema in the left upper lobe,

and significant airway thickening only in the lower lobes

for instance, with PET abnormalities in the apical segment

airway. Management more and more will need to be

specifically tailored to that case, taking into account also

other complex information such as genomic and environ-

mental information. Effects of therapy will also be followed

by complex image-based methods for that individual. As

we suggest, data management systems such as eidomics will

be extremely important in the future. This personalization

of healthcare delivery, as a result of image progress, such as

virtual bronchoscopy, represents a fundamentally import-

ant shift, complementing nicely similar shifts in genomic

research, and seems important for improving the precision

and outcome of healthcare delivery.

Conflicts of interest

G. McLennan and E.A. Hoffman are founders and part

owners of VIDA Diagnostics Inc.

References

1. Vining DJ, Liu K, Choplin RH, Haponik EF. Virtual bronchoscopy.

Relationships of virtual reality endobronchial simulations to actual

bronchoscopic findings. Chest 1996; 109: 549–553.

2. Quon A, Napel S, Beaulieu CF, Gambhir SS. ‘‘Flying through’’ and

‘‘flying around‘‘ a PET/CT scan: pilot study and development of 3D

integrated 18F-FDG PET/CT for virtual bronchoscopy and colon-

oscopy. J Nucl Med 2006; 47: 1081–1087.

3. Seemann MD, Schaefer JF, Englmeier KH. Virtual positron emission

tomography/computed tomography-bronchoscopy: possibilities,

advantages and limitations of clinical application. Eur Radiol 2007; 3:

709–715.

4. Ferguson JS, McLennan G. Virtual bronchoscopy. Proc Am Thorac

Soc 2005; 2: 488–491, 504–505.

5. McLennan G, Higgins W, Hoffman EA. Virtual bronchoscopy: im-

pact of the digital revolution. Pulm Perspect 2004; 21: 1–5.

6. Hoffman EA, Simon BA, McLennan G. State of the Art. A structural

and functional assessment of the lung via multidetector-row computed

tomography: phenotyping chronic obstructive pulmonary disease.

Proc Am Thorac Soc 2006; 3: 519–532.

7. Hoffman EA, McLennan G. Assessment of the pulmonary structure–

function relationship and clinical outcomes measures: quantitative

volumetric CT of the lung. Acad Radiol 1997; 4: 758–776.

8. National Institute of Standards and Technology. Imaging as a Bio-

marker: Standards for Change Measurements in Therapy. National

Institute of Standards and Technology, Gaithersburg, MD, USA, 2006.



9. Ukil S, Reinhardt JM. Smoothing lung segmentation surfaces in

three-dimensional X-ray CT images using anatomic guidance. Acad

Radiol 2005; 12: 1502–1511.

10. Finkelstein SE, Schrump DS, Nguyen DM, Hewitt SM, Kunst TF,

Summers RM. Comparative evaluation of super high-resolution CT

scan and virtual bronchoscopy for the detection of tracheobronchial

malignancies. Chest 2003; 124: 1834–1840.

11. Liewald F, Lang G, Fleiter T, Sokiranski R, Halter G, Orend KH.

Comparison of virtual and fiberoptic bronchoscopy. Thorac Cardio-

vasc Surg 1998; 46: 361–364.

12. Shitrit D, Valdsislav P, Grubstein A, Bendayan D, Cohen M, Kramer

MR. Accuracy of virtual bronchoscopy for grading tracheobronchial

stenosis: correlation with pulmonary function test and fiberoptic

bronchoscopy. Chest 2005; 128: 3545–3550.

13. Summers RM, Aggarwal NR, Sneller MC et al. CT virtual bronch-

oscopy of the central airways in patients with Wegener’s granulo-

matosis. Chest 2002; 121: 242–250.

14. Jones CM, Athanasiou T. Is virtual bronchoscopy an efficient diag-

nostic tool for the thoracic surgeon? Ann Thorac Surg 2005; 79: 365–

374.

15. Rooney CP, Ferguson JS, Barnhart W et al. Use of 3-dimensional

computed tomography reconstruction studies in the preoperative

assessment of patients undergoing balloon dilatation for tracheo-

bronchial stenosis. Respiration 2005; 72: 579–586.

16. Graham SM, McLennan G, Funk GF et al. Preoperative assessment

of obstruction with computed tomography image analysis. Am J

Otolaryngol 2000; 21: 263–270.

17. McAdams HP, Goodman PC, Kussin P. Virtual bronchoscopy for

directing transbronchial needle aspiration of hilar and mediastinal

lymph nodes: a pilot study. AJR Am J Roentgenol 1998; 170: 1361–

1364.

18. Higgins WE, Ramaswamy K, Swift RD, McLennan G, Hoffman EA.

Virtual bronchoscopy for three–dimensional pulmonary image

assessment: state of the art and future needs. Radiographics 1998; 18:

761–778.

19. Kiraly AP, Helferty JP, Hoffman EA, McLennan G, Higgins WE.

Three-dimensional path planning for virtual bronchoscopy. IEEE

Trans Med Imaging 2004; 23: 1365–1379.

20. Shinagawa N, Yamazaki K, Onodera Y et al. CT-guided trans-

bronchial biopsy using an ultrathin bronchoscope with virtual

bronchoscopic navigation. Chest 2004; 125: 1138–1143.

21. Asano F, Matsuno Y, Shinagawa N et al. A virtual bronchoscopic

navigation system for pulmonary peripheral lesions. Chest 2006; 130:

559–566.

22. Rooney CP, Wolf K, McLennan G. Ultrathin bronchoscopy as an

adjunct to standard bronchoscopy in the diagnosis of peripheral lung

lesions: a preliminary report. Respiration 2002; 69: 63–68.

23. Solomon SB, White P Jr, Wiener CM, Orens JB, Wang KP. Three-

dimensional CT-guided bronchoscopy with a real-time electromag-

netic position sensor: a comparison of two image registration methods.

Chest 2000; 118: 1783–1787.

24. Hautmann H, Schneider A, Pinkau T, Peltz F, Feussner H. Electro-

magnetic catheter navigation during bronchoscopy: validation of a

novel method by conventional fluoroscopy. Chest 2005; 128: 382–

387.

25. Mori K, Deguchi D, Akiyama K et al. Hybrid bronchoscope tracking

using a magnetic tracking sensor and image registration. Med Image

Comput Comput Assist Interv Int Conf Med Image Comput Comput

Assist Interv 2005; 8: 543–550.

26. Gildea TR, Mazzone PJ, Karnak D, Meziane M, Mehta AC. Elec-

tromagnetic navigation diagnostic bronchoscopy: a prospective study.

Am J Respir Crit Care Med 2006; 174: 982–989.

27. Venuta F, Rendina EA, De Giacomo T et al. Bronchoscopic proce-

dures for emphysema treatment. Eur J Cardiothorac Surg 2006; 29:

281–287.

28. Wan IY, Toma TP, Geddes DM et al. Bronchoscopic lung volume

reduction for end-stage emphysema: report on the first 98 patients.

Chest 2006; 129: 518–526.

29. Snell GI, Holsworth L, Borrill ZL et al. The potential for broncho-

scopic lung volume reduction using bronchial prostheses: a pilot study.

Chest 2003; 124: 1073–1080.

30. Tschirren J, Hoffman EA, McLennan G, Sonka M. Segmentation and

quantitative analysis of intrathoracic airway trees from computed

tomography images. Proc Am Thorac Soc 2005; 2: 484–487, 503–504.

31. Tschirren J, McLennan G, Palagyi K, Hoffman EA, Sonka M.

Matching and anatomical labeling of human airway tree. IEEE Trans

Med Imaging 2005; 24: 1540–1547.

32. Suter M, McLennan G, Reinhardt JM, Riker D, Hoffman EA.

Macro-optical color assessment of the pulmonary airways with sub-

sequent three-dimensional multidetector-x-ray-computed-tomogra-

phy assisted display. J Biomed Opt 2005; 10: 051703.

33. Suter M, Reinhardt J, Montague P et al. Bronchoscopic imaging of

pulmonary mucosal vasculature responses to inflammatory mediators.

J Biomed Opt 2005; 10: 034013.

34. Suter M, Tschirren J, Reinhardt J et al. Evaluation of the human

airway with multi-detector x-ray-computed tomography and optical

imaging. Physiol Meas 2004; 25: 837–847.

35. Suter MJ. Multi-modal Structural and Functional Description and

Analyses of the Pulmonary Airway Mucosa. University of Iowa, Iowa

City, IA, 2005.

36. Seibel EJ, Smithwick QY. Unique features of optical scanning, single

fiber endoscopy. Lasers Surg Med 2002; 30: 177–183.

37. Thiberville L, Moreno-Swirc S, Vercauteren T, Peltier E, Cave C,

Bourg Heckly G. In vivo imaging of the bronchial wall microstructure

using fibered confocal fluorescence microscopy. Am J Respir Crit

Care Med 2007; 175: 22–31.

38. Han S, El-Abbadi NH, Hanna N et al. Evaluation of tracheal imaging

by optical coherence tomography. Respiration 2005; 72: 537–541.

39. Hanna N, Saltzman D, Mukai D et al. Two-dimensional and

3-dimensional optical coherence tomographic imaging of the airway,

lung, and pleura. J Thorac Cardiovasc Surg 2005; 129: 615B–622B.

40. Tsuboi M, HayasBhi A, Ikeda N et al. Optical coherence tomography

in the diagnosis of bronchial lesions. Lung Cancer 2005; 49: 387–394.

41. McLennan G. Is the master clinician dead? Acad Med 2001; 6: 617–

619.

42. McLennan G, Namati E, Abdul-Malek K et al. The Eidome Project.

http://www.healthcare.uiowa.edu/InternalMedicine/Research/

tlirp/Eidome/Default.htm, 2006.



virtual bronchoscopy

Documents