M

Ya

b

a

AA

KMHSHQ

1

bfbadmtii[usbbf(setamf

(

0d

Sensors and Actuators A 188 (2012) 450– 455

Contents lists available at SciVerse ScienceDirect

Sensors and Actuators A: Physical

j ourna l h o me pa ge: www.elsev ier .com/ locate /sna

EMS scanner based handheld fluorescence hyperspectral imaging system

oumin Wanga, Sheldon Bishb, James W. Tunnellb, Xiaojing Zhangb,∗

Department of Electrical and Computer Engineering, University of Texas at Austin, 1 University Station C0803, Austin, TX 78712, USADepartment of Biomedical Engineering, University of Texas at Austin, 1 University Station C0800, Austin, TX 78712, USA

r t i c l e i n f o

rticle history:vailable online 14 December 2011

a b s t r a c t

We demonstrate a micromirror based hand-held hyperspectral fluorescence imaging microsystem forskin cancer detection, where multiple narrow-band spectral images can be captured simultaneouslyacross the area under examination. In addition to the advantages of combined functional imaging

eywords:icroelectromechanical system (MEMS)andheld instrumentskin canceryperspectral imaginguantum dots (QDs)

and spectroscopy, the system demonstrates the fast scanning over large field-of-view (FOV) pro-vided by a CMOS compatible 2-axis microelectromechanical system (MEMS) scanning mirror in theprobe. Spectral information from 15 wavelengths was acquired simultaneously using a multichan-nel photomultiplier tube (PMT), rendering a spectral resolution of 20 nm. FOV up to 0.8 cm × 0.5 cmwith 50 �m lateral resolution was acquired, with a hyperspectral imaging rendering rate of 2 s perframe.

Published by Elsevier B.V.

. Introduction

Hyperspectral imaging, which integrates the advantages ofoth conventional imaging and spectroscopy, were first designedor remote sensing [1], while now finds diverse applications iniomedical imaging field. Portable hyperspectral screening devicesre highly desirable for sensitive biopsy-free characterization ofiseases such as epithelial cancers in situ and precision guidedicrosurgery. Spectral resolution and frame rate in such minia-

urized imaging devices are in high demand, which has led tontensive research on developing novel hand-held multi-channelmaging and spectroscopy instruments using microtechnologies2]. Traditional hyperspectral imaging techniques involve using liq-id crystal and acousto-optic tunable filters [3], Fourier-transformpectrometry [4] or spectral–temporal scanning [5] to acquireoth morphological and spectral information, while capturingoth spatial and spectral data within one frame is highly pre-erred to reduce processing time. Acousto-optic tunable filterAOTF) or liquid crystal tunable filter (LCTF) were adopted inpectral scanning switching the wavelength selection [6,7]. How-ver, the imaging rate is limited by the switching rate, whilehere is also trade-off between the spectral image quality and the

cquisition time. Currently most advanced hyperspectral imagingicroscope on market could acquire hyperspectral imaging at 5

rames per second (fps) with 512 × 512 pixel resolution [8], but

∗ Corresponding author.E-mail addresses: jtunnell@mail.utexas.edu, John.Zhang@engr.utexas.edu

X. Zhang).

924-4247/$ – see front matter. Published by Elsevier B.V.oi:10.1016/j.sna.2011.12.009

with a bulky size and a costly price. Integrating the CMOS compati-ble dual-axis micromirror, we have already demonstrated multiplemicro-imaging modalities, including a handheld forward-imagingconfocal microscope capable of sub-micrometer lateral resolutionfor early cancer detection [9] and high-speed OCT for cardiovas-cular imaging [10]. In this paper, we demonstrate a new systemcapable of doing fast hyperspectral imaging to acquire spectralinformation across wide spectrum simultaneously with adjustablespatial resolution towards applications in MEMS based com-pact in vivo hyperspectral microscopy for early epithelial cancerdiagnosis.

2. Methods

2.1. MEMS scanning mirror

The MEMS micromirror (Fig. 1) was the core componentenabling the imaging system [11]. The micromirror has a diame-ter of 1024 �m, sitting on the microchip which has a dimension of2.8 mm × 2.8 mm. When applied high voltages on the two orthogo-nal of the comb-drive structure, the staggered vertical comb-drivestructure of the chip actuates the micromirror within the gimbalstructure to rotate along the two perpendicular axes. The pri-mary mechanical resonant frequencies of the fabricated MEMSmicromirror were 2.57 kHz and 1.2 kHz for the inner and outer rota-tion axis respectively. Lissajous scanning can be achieved through

applying sinusoidal high voltage AC actuating signal onto the twoaxes of the MEMS micromirror.

The fabricated MEMS scanning mirror was mounted onto aprinted circuit board (PCB) to ease the wire bonding, power

Y. Wang et al. / Sensors and Actuators A 188 (2012) 450– 455 451

F . (a) Ta outer

dhwa

MAo

Fm

ig. 1. Scanning element in the handheld fluorescent hyperspectral imaging systemnd rotor comb fingers on the inner axis. (c) View of the comb-drive actuator on the

elivery and packaging, as shown in Fig. 1(d). The MEMS chipas the wire-bondings onto it from 4 copper pads. Power deliveryires were soldered onto the board to enable high voltage MEMS

ctuation.

To maximize the field-of-view (FOV) of the imaging system, the

EMS scanner was running under the Lissajous scanning pattern. pair of 75 V peak-to-peak sinusoidal AC voltage with an offsetf 50 V was used to actuate the MEMS micromirror, actuating the

ig. 2. Schematic of the handheld MEMS hyper-spectral imaging system, the packagingirror and the data acquisition from 16Ch PMT.

op view of the micromirror. (b) View of the quality of alignment between the stator axis. (d) Photograph of the MEMS micromirror mounted on a printed circuit board.

MEMS mirror to reach the maximum optical deflection angles at28◦ and 25◦ respectively for the inner and outer axes.

2.2. Handheld hyperspectral imaging instrumentation

This fluorescence hyperspectral imaging system (Fig. 2) useda 488–514 nm argon laser (Coherent Medical, Baltimore, MD) asexcitation source. The illumination light was coupled into a single

design of the probe, and the signal flow for the actuation of the MEMS scanning

452 Y. Wang et al. / Sensors and Actua

Ftt

mie

dnElds0lnmmotap

NAA1(tasO

cmatt

d3ZahC

fi

ig. 3. Photograph of the assembled handheld MEMS hyper-spectral imaging sys-em shows the case cover, and the main case with the MEMS scanning mirror andhe optical elements inside.

ode optical fiber (SMF) which transmits the excitation laser lightnto the handheld probe, where a lens fixed to the terminated fibernd collimates the beam.

Guided by the SMF which was attached on the customeresigned handheld probe using a pigtailed connector, this illumi-ation was transmitted through a hot (dichroic) mirror (NT69-865,dmund Optics, Barrington, NJ) with 520 nm cutting-off wave-ength. After passing through the dichroic, the illumination beam iseflected onto the surface of the microelectromechanical (MEMS)canning mirror before being focused by the objective lens (AC127-50-A, Thorlabs, Jessup, MD) onto the sample. The fluorescence

ight emitted from the quantum dots (QDs) sample was simulta-eously de-scanned by the MEMS mirror and reflected by the hotirror onto the collection arm, where it is coupled into a multi-ode SMA (SubMiniature version A) collection fiber leading out

f the probe. The collection fiber terminates on a spectral acquisi-ion module, where the collection light was spectrally dispersed by

triangular prism onto the photocathodes of a 16 Channel arrayhotomultiplier tube (PMT) (H7260, Hamamatsu, Japan).

Data acquisition was performed using a 16 Channel analog inputI PXIe-6358 data acquisition card (DAQ) (National Instruments,ustin, TX), in concert with a LabView® (National Instruments,ustin, TX) and Matlab® based software to simultaneously acquire5 bands of fluorescence signal. The remaining one analog inputAI) channel was used to synchronize the acquisition timing withhe scan signal for image rendering. The MEMS scanning mirror isctuated in resonant modes on both axes upon applying the sinu-oidal scan signal from function generators (Tektronix, Beaverton,R).

As shown in Fig. 3, the handheld imaging probe consists of a caseover and the main case which incorporates the MEMS scanningirror and the optical elements including lenses, reflection mirrors

nd the short-pass dichroic mirror. The handheld probe interfaceshe single mode excitation laser fiber, the SMA collection fiber andhe power delivery wires through three I/O ports.

The handheld probe measures 13 cm × 11 cm × 3.7 cm in outerimensions, the pieces were first designed and modeled using theD modeling software SolidWorks® before physically crafted by aCorp® 3D printing technique using the material of plastic powder,nd adhered by cyanoacrylate. Peripheral serving facilities for the

andheld imaging probe including the dispersing optics and the 16hannel PMT were mounted onto an optical board.

The optical adjustments for the alignment of the SMA collectionber were made possible through two degrees of freedom design.

tors A 188 (2012) 450– 455

First a one dimensional lateral adjustment screw was used at thecollection fiber port for vertical position adjustment. The lateraldirection adjustment was realized through the use of an adjustablemirror stage for the mirror reflecting the excitation laser beam,which has pitch and roll adjustment screws on it. Thus focusing ofthe emitted fluorescence light into the core of the collection fiberwas guaranteed. While tuning the reflection angle of the first mirrorto adjust the focusing of fluorescence beam, the position and spatialangle of the MEMS mirror were also adjusted by tuning the stagewhere the PCB holding MEMS mirror was mounted on, to ensurethe excitation beam is hitting the center of it.

3. Results and discussion

3.1. Optical design simulations

Fig. 4(a) shows the schematic of the optical layout of the opticalscanning imaging system using optical simulation software CODEV®. With the total optical scanning angle of the MEMS micromir-ror of 20◦, the field of view of the simulated optical system is13.4 mm × 13.4 mm. Fig. 4(b) shows the simulated distortion in thelateral field of view, which indicates the optical path difference(OPD) with both tangential and sagittal directions within that fieldof view for three wavelengths: 520 nm, 580 nm and 610 nm.

The selection of the simulation wavelengths are based on theemission spectrum of the sample we use.

Lateral resolutions are evaluated in Fig. 4(c). We plot thediffracted Gaussian beam spot on the focal plane of the optical sys-tem, with reference to different scanning angles. It shows the airydisk shapes over the entire imaged field of view.

The quantitative determination of the optical system’s resolu-tion is shown as well, indicating the minimum spot size of 19.7 �mat the intersection position of the main optical axis and the focalplane, while the maximum spot size is obtained to be 36.90 �mwhen the MEMS micromirror is rotated by 10◦. The position offsetcaused by the wavelength difference of illumination was markedby different color patterns.

3.2. Handheld hyperspectral imaging system calibration

The hyperspectral performance of this system is demonstratedby the imaging of a three layered polydimethylsiloxane (PDMS)phantom sample with three quantum dots stamps on each layer,QDs patterns are located inside the phantom sample at differentdepths, varying from 600 �m at the left column to 200 �m at theright end column. The fluorescence images of the phantom sam-ple under 15 wavelengths acquired simultaneously are shown inFig. 5. Yellow and red QDs patterns have much stronger fluores-cence signal yield out at the shallower columns, while the greenQDs pattern does follow this rule due to the distribution unifor-mity. (For interpretation of the references to color in this figure,the reader is referred to the web version of this article.)

Micro-contact printing (�CP) was used to pattern the quan-tum dots on the surfaces of the multi-stack PDMS thin layers. Eachspin-coated PDMS thin-layer is 200 �m in thickness, infused withtitanium dioxide to simulate the scattering coefficients of the bio-logical tissue. The sample contained a 3 × 3 quantum dots patternarray, each depth containing 3 different colors on it. The 3 kindsof quantum dots have emission peaks of 520 nm (green), 580 nm

(yellow), and 620 nm (red) with a full width half maximum ofapproximately 40 nm. This 3 × 3 grid of quantum dot stamps isorganized such that color is sequestered by row, and stamp depth(from layering) by column.

Y. Wang et al. / Sensors and Actuators A 188 (2012) 450– 455 453

F ) 2D la( tical

i

3

oqsiectqtsia

Fi

ig. 4. The optical layout of the scanning objective system simulated in CODE V® . (ac) Simulated spot diagram for three wavelengths with reference to the scanning ops referred to the web version of this article.)

.3. Handheld hyperspectral imaging results

The images with 15 wavelengths of data obtained simultane-usly, looking at a FOV of a 1.1 cm × 0.7 cm area of the PDMSuantum dots sample were acquired, at a rate of 4 s per hyper-pectral frame. Using this hyperspectral image array as shownn Fig. 5, the spatial information for the phantom sample underach wavelength could be examined, while individual pixelsould also be analyzed for their spectral content. The respec-ive collected fluorescence spectrum of the red, yellow and greenuantum dots are easily selected by examining the spectrum of

he pixels demarking their spatial location as marked in red ashown in Fig. 6. (For interpretation of the references to colorn this figure, the reader is referred to the web version of thisrticle.)

ig. 5. (a) The photograph of the QD sample under UV illumination, the area inside the dmages of the phantom sample under 15 wavelengths acquired simultaneously.

yout of imaging optical system. (b) Simulated distortion in the lateral field of view.angle. (For interpretation of the references to color in this figure legend, the reader

Using the images of one peak wavelength band for each color,the pseudo-color image was merged and rendered, which showsgood preserving of features when comparing to the mosaic imagefrom a commercial microscope (Fig. 6(b)).

Upon the observation of the collected spectra from the handheldhyperspectral imaging system with comparison with that from acommercial standard spectrometer, the spectral broadening effectof the fluorescence signal can be seen. This spectral broadeningfrom our system was due to the low magnification of the collec-tion optics, combined with the relatively large size of the collectionfiber (1 mm). The large collection fiber image on the PMT cathode

plane was necessary to increase the signal to noise ratio (SNR) toacceptable levels, although at the expense of spectral resolution. QDsignal from red, yellow and green patterns were measured acrossthe visible spectrum, demonstrating a spectral resolution of 20 nm

ashed box is that imaged from the hyperspectral imaging system. (b) Fluorescence

454 Y. Wang et al. / Sensors and Actuators A 188 (2012) 450– 455

Fig. 6. Imaging PDMS phantom with embedded multicolor quantum dots. (a) Pseudocoland 620 nm, respectively. (b) Mosaic fluorescence image taken using Olympus BX51 micr

Fig. 7. Comparison of the spectrum acquired for the quantum dots sample fromhandheld hyperspectral imaging probe and commercial spectrometer. (a) Spectrumof the 3 color quantum dots on the red marker positions of Fig. 4, acquired by hand-held FSI system; (b) spectrum of the quantum dots at same position, acquired by acommercial standard spectrometer (USB 4000, Ocean optics).

or image merged from three channels with central wavelength of 520 nm, 580 nmoscope. Scale bars: 1 mm.

(Fig. 7). (For interpretation of the references to color in this figure,the reader is referred to the web version of this article.) Due to thedichroic mirror’s 500 nm cutoff and transient slope, the green spec-trum is slightly attenuated on its ‘blue’ side, giving the impressionof a peak at 520 nm. A commercial spectrometer (Ocean Optics,Dunedin, FL) was used for comparison to the spectral performanceof our handheld hyperspectral imaging system. The spectra fromboth the commercial spectrometer and our system were normal-ized for comparison.

4. Conclusions

In this paper, we demonstrated a handheld hyper-spectral imag-ing system using a MEMS scanning micromirror. Spatial FOV of1.1 cm × 0.7 cm with optical resolution of 50 �m are demonstrated.Larger field sizes of up to 1.2 cm have been demonstrated in pre-vious experiments [12]. Hyperspectral resolution of 20 nm from490 to 730 nm, with a fast imaging speed of 2 s per hyperspec-tral frames were obtained for images of a quantum dots infusedPDMS phantom sample. Preliminary characterization results frommulti-color layered phantoms show the great potential of in vivohyperspectral imaging for this MEMS enabled probe for biologicaland clinical applications. In addition, the handheld MEMS basedscanning spectrometer can be potentially suitable for detectingauto-fluorescence spectrum of skin cancer tissue. This endogenousfluorescence spectroscopy could be a viable technique to detectdysplasia in the cervix, ovary and the esophagus. Prominent tis-sue fluorophores with excitation–emission peaks in the visibleregion include nicotinamide adenine dinucleotide (NADH), colla-gen and flavin adenine dinucleotide (FAD). Power efficiency can beenhanced with proper excitation and emission band selected.

Acknowledgments

The financial support of this research is by National ScienceFoundation under grant #99150 and National Institutes of Health

Actua

(Tac

R

[

[

[

B

YJEPAmau

the British Council Early Career RXP Award in 2008, NSF Faculty Early Career Devel-opment Program (NSF CAREER) Award in 2009–2014, DARPA Young Faculty Award in

Y. Wang et al. / Sensors and

Grant No. R01 CA132032, Tunnell) are gratefully acknowledged.he authors wish to thank Microelectronics Research Center (MRC)t UT, Austin for providing facilities for microdevice fabrication andharacterization.

eferences

[1] J.A. Richards, X. Jia, Remote Sensing Digital Image Analysis: An Introduction,Springer Verlag, 2006.

[2] Y.G. Patel, K.S. Nehal, I. Aranda, Y. Li, A.C. Halpern, M. Rajadhyaksha, Confocalreflectance mosaicing of basal cell carcinomas in Mohs surgical skin excisions,Journal of Biomedical Optics 12 (2007) 034027.

[3] H.R. Morris, C.C. Hoyt, P.J. Treado, Imaging spectrometers for fluorescence andRaman microscopy: acousto-optic and liquid crystal tunable filters, AppliedSpectroscopy 48 (1994) 857–866.

[4] A.R. Harvey, D.W. Fletcher-Holmes, Birefringent Fourier-transform imagingspectrometer, Optics Express 12 (2004) 5368–5374.

[5] J.E. Murguia, T.D. Reeves, J.M. Mooney, W.S. Ewing, F.D. Shepherd, A.K. Brodzik,A compact visible/near infrared hyperspectral imager, Proc. SPIE (2000)457–468.

[6] V. Ntziachristos, J. Ripoll, L.V. Wang, R. Weissleder, Looking and listening tolight: the evolution of whole-body photonic imaging, Nature Biotechnologyvol. 23 (2005) 313–320.

[7] R. Lansford, G. Bearman, S.E. Fraser, Resolution of multiple green fluorescentprotein color variants and dyes using two-photon microscopy and imagingspectroscopy, Journal of Biomedical Optics 6 (2001) 311.

[8] LSM 510 META Product Brochure, http://www.zeiss.com.[9] K. Kumar, R. Avritscher, Y. Wang, N. Lane, D.C. Madoff, T.K. Yu, J.W. Uhr, X.

Zhang, Handheld histology-equivalent sectioning laser-scanning confocal opti-cal microscope for interventional imaging, Biomedical Microdevices 12 (2010)223–233.

10] K. Kumar, J.C. Condit, A. McElroy, N.J. Kemp, K. Hoshino, T.E. Milner, X. Zhang,Fast 3D in vivo swept-source optical coherence tomography using a two-axisMEMS scanning micromirror, Journal of Optics A: Pure and Applied Optics 10(2008) 044013.

11] K. Kumar, R. Avritscher, D.C. Madoff, X. Zhang, Handheld single-cell-layer opti-cal sectioning reflectance confocal microscope for interventional imaging, in:The 29th Conference on Lasers and Electro Optics (CLEO), Baltimore, MD, June1–5, 2009.

12] Y. Wang, S. Bish, J.W. Tunnell, X. Zhang, MEMS scanner enabled real-time depthsensitive hyperspectral imaging of biological tissue, Optics Express 18 (2010)24101–24108.

iographies

oumin Wang received the B.Eng. degree in Electronics Engineering from Shanghaiiao Tong University, China, in 2008 and the M.S. degree in Electrical and Computerngineering from the University of Texas at Austin, in 2010. He is now pursuing his

h.D. degree in Electrical and Computer Engineering from the University of Texas atustin under the supervision of Dr. John X.J. Zhang. His research focuses on opticalicroelectromechanical systems (MEMS) design, simulation, fabrication and char-

cterization for several applications including handheld confocal in vivo l imagingsing Optical MEMS device and hyperspectral fluorescence imaging.

tors A 188 (2012) 450– 455 455

Sheldon Bish holds a B.S. in Biomedical Engineering from the University of Connecti-cut in 2007. He is now pursuing his Ph.D. in Biomedical Engineering at the Universityof Texas at Austin under the supervision of James W. Tunnell. His research focus ison diffuse optical spectroscopy towards the imaging of tissues for in vivo skin cancercharacterization and detection.

James W. Tunnell is an Associate Professor in the Department of Biomedical Engi-neering at the University of Texas at Austin. He earned a B.S. in Electrical Engineeringfrom the University of Texas at Austin in 1998 and a Ph.D. in Bioengineering fromRice University in 2003. He was awarded a National Research Service Award from theNIH to fund his postdoctoral fellowship in the Spectroscopy Laboratory at the Mas-sachusetts Institute of Technology from 2003 to 2005. He joined the faculty of theUniversity of Texas in the fall of 2005. Dr. Tunnell’s research focuses in the broad fieldof biomedical optics with a specific focus on using optical spectroscopy and imag-ing for disease diagnosis and treatment, particularly that of cancer. Dr. Tunnell hasreceived the following awards/honors: Outstand BME Graduate Alumnus from RiceUniversity (2010), Coulter Fellow (2010), Ralph E. Powe Junior Faculty EnhancementAward from the Oak Ridge Associated Universities (2007), Early Career Award fromthe Wallace H. Coulter Foundation (2008, 2006), National Research Service Awardfrom the NIH (2004), and Best Basic Science Paper from the American Society forLaser Medicine and Surgery (2000). He has published over 30 referred journal arti-cles, presented at more than 70 international and national conferences, and editedone book. He is an Associate Editor for the Annals of Biomedical Engineering. He hasserved on the program committees for CLEO, OSA, and IEEE-LEOS, and he is the Gen-eral Chair of 2012 CLEO annual meeting. He is a member of OSA, ASLMS, IEEE-EMBS,and BMES.

Xiaojing Zhang is an Associate Professor at the University of Texas of Austin (UT,Austin) in the Department of Biomedical Engineering, in joint affiliations withInstitute for Cellular and Molecular Biology (ICMB), Microelectronics Research Cen-ter and Texas Materials Institute. He received his Ph.D. from Stanford University,California in 2004, and was a Research Scientist at Massachusetts Institute ofTechnology (MIT), Cambridge, before joining the faculty at UT, Austin in 2005.Zhang’s research focuses on exploring bio-inspired nanomaterials, scale-dependentbiophysics, and nanofabrication technology, towards developing new diagnosticdevices and methods on probing complex cellular processes and biological networkscritical to development and diseases. Both multi-scale experimental and theoreticalapproaches are combined to investigate fundamental force, flow and energy pro-cesses at the interface of engineering and biomedicine. In particular, his laboratoryis pioneering the development of integrated photonic microsystems (MEMS, micro-electro-mechanical systems), semiconductor chips and nanotechnologies criticalto healthcare, defense and environmental applications. He has published over 120peer reviewed papers and proceedings, presented over 45 invited seminars world-wide, and filed over 15 US patents. His research findings have been highlighted inmany public media, and were licensed to two companies. He has organized manymajor conferences and sessions in the area of MEMS/BioMEMS, nanotechnologiesand biomedical engineering. In addition to being the Principle Investigator of manymajor grants from U.S. federal agencies such as NIH, NSF and DARPA, Dr. Zhang wasalso recipient of many prestigious awards, including: the Wallace H. Coulter Founda-tion Early Career Award for Translational Research in Biomedical Engineering in 2006,

2010, and one of 85 invitees from both academia and industry under the age of 45to attend U.S. National Academy of Engineering, Frontiers of Engineering (NAE-FOE)program in 2011.