Review of digital holographicmicroscopy for three-dimensionalprofilingand trackingXiao YuJisoo HongChanggeng LiuMyung K. KimReview of digital holographic microscopy forthree-dimensional profiling and trackingXiao Yu,* Jisoo Hong, Changgeng Liu, and Myung K. KimUniversity of South Florida, Digital Holography and Microscopy Laboratory, Department of Physics, Tampa, Florida 33620Abstract. Digital holographicmicroscopy(DHM)isapotent tool toperformthree-dimensional imagingandtracking. We present a review of the state-of-the-art of DHM forthree-dimensional profiling and tracking withemphasisonDHMtechniques, reconstructioncriteriafor three-dimensional profilingandtracking, andtheirapplicationsinvariousbranchesof science, includingbiomedical microscopy, particleimagingvelocimetry,micrometrology, and holographic tomography, to name but a few. First, several representative DHM configu-rations are summarized and brief descriptions of DHM processes are given. Then we describe and compare thereconstruction criteria to obtain three-dimensional profiles and four-dimensional trajectories of objects. Details ofthesimulatedandexperimental evidencesofDHMtechniquesandrelatedreconstructionalgorithmson par-ticles,biological cells,fibers,etc.,withdifferentshapes,sizes,andconditionsarealsoprovided.Thereviewconcludes with a summary of techniques and applications of three-dimensional imaging and four-dimensionaltracking by DHM. The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction ofthis work in wholeor inpart requires full attribution of the original publication,including its DOI. [DOI:10.1117/1.OE.53.11.112306]Keywords: digital holography; microscopy; three-dimensional profiling; four-dimensional tracking; microparticles; nanoparticles;microfibers; cells.Paper 140175SSreceived Jan. 29, 2014; revised manuscript received Feb. 28, 2014; accepted for publication Mar. 4, 2014; publishedonline Apr. 3, 2014.1IntroductionNowadays, three-dimensional (3-D) profilingandtrackingmicroscale objects have been receiving much attentiondue to their wide applications. For example, microsphere ormicrobubble counting and locating in a 3-D space occur inthe fields of microfluidics, suspension rheology, crystalliza-tion, etc.111Many microorganisms swim in 3-D and helicalpaths. Thus, measuring their 3-D trajectories in time is essen-tial to obtain detailed information on biophysical processes,such as motile behavior and dynamic performance.1214Quantitativeanalysesofcancercell locomotionandshapechange in a 3-D environment reveal the biological character-isticsfor clinical need.15203-Dimagingofrandomlyori-entedmicrofibers andtheir interactions withsurroundingfree-swimmingcells opens upnewperspectives incaseswhere quantitative measurement of characteristics (size, length,orientation, speed, displacement, etc.) is of great interest.21Various approaches have been demonstrated for 3-Dprofilingandtrackingof micro-sizedobjects, andopticaltechniques and numerical localization algorithms have beenwidely chosen as remarkable tools since they have the advan-tages of being full-field, label-free, noncontact, noninvasive,etc.2243Digital holographic microscopy (DHM) is an emerg-ingtechnologyofanewparadigmingeneralimagingandbiomedical applications.44,45Various techniques of DHM,includingoff-axisDHM, in-lineDHM, quantitativephasemicroscopybydigital holography(DH-QPM), etc., havebeenproventobepotent toolstoprofileandtrackmicro-sizedobjectsin3-Dvolumes. Inconventional microscopytechniques, only two-dimensional focused images on a fixedplane can be recorded, while information not in the focal planewouldbepermanentlymissed. However, DHMprovidesatool torefocusanobject andisabletorecordahologramcontainingall thereal-time, 3-Dstructuresofanobject inthe absence of mechanical focus adjustment. And the infor-mation is available in digital form for postprocessing. DHMhas been shown to be the key to 3-D particleimage veloc-imetry (PIV)4650since the conventional PIV techniques havetheinherent limitations of thindepthof field, instrumentcomplexity, and impossibility of real-time imaging.5153However, DHM is able to overcome these and has the sim-plicity of the microscope and sample preparation, maximuminformation and resolution, reduction in time and dataamount, etc. Aberrations and background distortions ofthe optical field can be minimized by available DHM tech-niques. The DHM for 3-D profiling and tracking generally isa two-step process: first a hologram is recorded digitally, andthenthe hologram is numerically reconstructed in differentdepths to yield an image of the object by various numericaldiffractionmethods, suchasangularspectrummethod,44,45Kirchhoff-Helmholtz transform,54Fresnel transform,55,56Huygens convolution,44,45etc. This reconstruction resultsin complex field of object, and one can extract its amplitudeor phase profile to represent the object. Investigations of thethird dimension based on the reconstructed data are then per-formed to determine focal planes and complete the 3-D pat-tern recognition. For a moving object, the reconstruction ofsuccessiveholographicdatayieldsacompletefour-dimen-sional space-timerecordofthedynamicprocess. DHMisdemonstrated to have the capacity of monitoring the 3-D dis-tributionandmotionpatternof particles, livingcells, andfiberswithdifferent shapes(spherical, needleshaped, andrandomly oriented),size(fewto hundredmicrometer),andconditions (static, suspended, and flow-through) in real time.Description of several representative DHM techniques isintroducedinSec.2,andtheholographicinformationpro-cessingmethodsfor 3-Dprofilingandtrackingaregiven *Address all correspondence to: Xiao Yu, E-mail:xyu4@mail.usf.eduOptical Engineering 112306-1 November 2014 Vol. 53(11)Optical Engineering 53(11), 112306 (November 2014)REVIEWin Sec. 3. Wide and active field of applications on 3-D profil-ingandtrackingofdifferent samples, includingmicropar-ticles, bubbles,biologicalcells,microfibers,etc., byDHMtechniquesareintroducedinSec. 4, andtheconclusionisprovided in Sec. 5. This review has an emphasis on applica-tions of DHM in the field of 3-D profiling and tracking and,therefore,omits somemajorareas,suchas digital hologra-phy principle and development, theoretical studies of digitalholography,specialdigitalholography techniques,etc.2DHM ConfigurationsAbasicDHMsetupconsistsofanilluminationsource, aninterferometer, a digitizing camera, and a computer with nec-essary programs. Most often a laser is used for illuminationwiththenecessarycoherencetoproduceinterference. Formultiwavelengthtechniques, twoor more different laserscan be coupled into the interferometer, or a tunable laser canbeemployed.Therearealsolow-coherencetechniquesforthepurposeofreducingspeckleandspuriousinterferencenoise or generating contour or tomographic images. Even anLED typically has 30 m or so coherence length, which canbesufficient forholographicmicroscopy.2.1In-Line DHMIn this review, we refer to Gabor holography as in-line holog-raphy.57In-line DHM is a type of microscopy without objec-tive lenses,and as illustrated in Fig.1, a singlelight beamdirected onto a pinhole of a diameter of the order of a wave-length illuminates the object, typically several thousandwavelengthsawayfromthepinhole, andtheobject beamis the part of the incident light that is scattered by the objectand the unscattered remainder is taken as the reference beam.If the light source is coherent, the pinhole is not necessary.The object field is automatically in alignment with the refer-ence beam, and the interference of the object and referencebeams results in the holographic diffraction pattern, which isrecorded by CCD camera and then transferred to a computerfornumericalreconstruction.Byremovingbackgroundeffects first andthenrecon-structingtheobject fieldat different planes, a3-Dimagecan be built up from a single two-dimensional image, and theentire hologram pixel count is utilized, which also leads toshorter minimum distances for reconstruction and higher res-olutionof theresultant image. Withthesimplicityof theapparatus and large depth of field, the in-lineDHM is par-ticularly useful for particle image analysis. In conjunction withreconstruction algorithm, the 3-D position of a particle can bedetermined andthen appliedtoparticle velocimetry,such astracking small particles or swimming cells in a liquid flow.2.2Off-Axis DHMIn an off-axis DHM setup, a Mach-Zehnder interferometer ischosensince it offers flexibilityinalignment, especiallywhen microscopic imaging optics is used. One microscopeobjective lens (MO) is needed for object magnification andanother one is used in the reference arm to match the curva-tures of the object and reference wave fronts. We illustratedtheoff-axis holographicmicroscopysetupinFig. 2. Theobject arm contains a sample stage and an MO that projectsamagnifiedimageoftheobject ontoaCCDcamera. Thereference arm similarly contains another MO, and the refer-enceandobject wavesareoffset byanangletoavoidtheoverlapof thereferenceandthetwinimages, sothat theholographic interference pattern contains fringes due to inter-ferencebetweenthediffractedobjectfieldandtheoff-axisreference field. The captured holographic image is numeri-callyconvertedintoFourier domaintoobtaintheangularspectrum,44,45andaspatial filter isthenappliedtoretainthereal imagepeakalone. Thefilteredangular spectrumis propagatedtoappropriate distance and, byaninverseFourier transform, reconstructed as an array of complexnumbers containing the amplitude and phase images ofthe sample. Both amplitude and phase profiles can be furtheranalyzed to quantitatively determine the positions of objectsina 3-Dspace.2.3QuantitativePhase Microscopy by DigitalHolographyOff-axis DHMis a very effective process for achievinghigh-precision QPM since it allows measurement of opticalFig. 1In-line digital holographic microscopy (DHM) setup. MO, micro-scope objectives; P, pinhole; S, sample object.Fig. 2Off-axis DHM setup. M's, mirrors; BSs, beam splitters; MOs, microscope objectives; S, sampleobject.Optical Engineering 112306-2 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingthickness with nanometer-scale accuracy by single-shot,wide-fieldacquisition, andit yieldsphaseprofileswithoutsome of the complications of other phase imaging methods.Thephaseimageisimmediatelyanddirectlyavailableoncalculating the two-dimensional complex array of the holo-graphicimage, andthephaseprofileconveysquantitativeinformation about the physical thickness and index of refrac-tion of cells. We have recently utilized DH-QPM to study thewrinklingofasiliconerubberfilmbymotilefibroblasts.58The wrinkle formation has been visualized, quantitativemeasuresofsurfacedeformationhavebeenextracted, andcellular tractionforcehas beenestimatedinadirect andstraightforward manner. A nonwrinkling substrate, collagen-coatedpolyacrylamide(PAA)wasalsoemployedtomakedirect measurement of elastic deformations, albeit at discretelocations.59The Youngs modulus of PAA can be adjusted bycontrollingtheconcentrationsof themonomer andcross-linker. DH-QPM, withitscapacityofyieldingquantitativemeasures of deformationdirectly, has beenemployedtomeasuretheYoungsmodulusof PAA,60whichprovideda very effective process for achieving high-precision quanti-tative phase microscopy. We have utilized DH-QPM to studymotile fibroblasts deforming PAA gel,61where the cell-sub-strateadhesionhasbeenvisualizedandquantitativemea-sures of surface deformation have been extracted. Thesubstrate stiffness and quantitative measures of substratedeformation have been combined to produce estimates of thetraction forces and characterize howthese forces varydependingon thesubstraterigidity.2.4Twin-Beams DHMThetwin-beamsDHMsetupisillustratedinFig.3.62Twobeams, coming fromthe same laser source and slightlyoff-axis, enter into the20 microscope objective and resultin an image on a CCD plane. Theoretically, the two beamshavethesamefocal plane; however, thetwinbeams canexperience aberrations resulting in some focus-shift. AsshowninFig. 3, eachparticle of the sample appears astwo projections on the CCD array. The tracking is performedbyevaluatingthetwoout-of-focusprojectionsof thepar-ticlesduetothetwin-beamsontotheCCDplane, andtheseparation between the two projections is a function ofthelongitudinal positionof theparticle. QPMimages ofthemicro-objectscanbedirectlyobtainedfromtheholo-grams recordedby thesameconfiguration.3Numerical Focus Methods for Three-DimensionalProfiling and Tracking by DHM3.1Quantificationof ImageSharpness and PeakSearchingTolocalizeanobject ina3-Dvolume, anautomateddataanalysisalgorithmtoaccuratelyidentifydepthpositionisrequired. Forexample, pureamplitudeobjectslocatedoutoffocal planeappearasgraypatcheswithout sharpstruc-tures, whilepurephaseobjects showminimumvisibilitywhen they are in focus.6365Thus, focusing metrics based onthe quantification of image sharpness are applied by search-ing on the reconstructed images for the planes that containthe objects with the peak and sharpest details to establish theall-in-focusprofileanditscorrespondingdepthmap.66Ateachpixel, theintensityvariationinlongitudinal directionis investigated to find out the peak intensity. The peak inten-sity value becomes a pixel of an all-in-focus profile and itslocationinlongitudinal directionis recordedas a corre-sponding depth map value. The combination of resultant all-in-focusprofile anddepthmapenables onetoproducethe3-D visualization of objects, and object properties can thenbeextracted.We recently utilized off-axis DHM and numerical autofo-cusingalgorithmbasedonpeaksearchingtoimage sus-pended polymer microspheres (9.6 min diameter) insuspension.21Figure4(a)presentsthehologramofmicro-spheresinsuspensioncapturedbyoff-axisDHM. Angularspectrummethod44,45was applied and the reconstructedamplitude image was shown in Fig. 4(b), where several in-focus and out-of-focus microspheres were seen at thisreconstructionplane. Thefieldof viewwas 90 90m2with 464 464pixels. The microsphere focused the incom-inglight andformedabright spot alongtheoptical path.Therefore,thepattern of the microsphere appearedto havea maximum intensity at the center near the in-focus plane andthe surrounding was dark. Then, the autofocusing algorithmbased on peak searching was applied to identify the in-focuspositionof eachparticle. Thehologramwas numericallyreconstructed in longitudinal direction Z from 100 to100m, in 2-msteps. The autofocusing algorithmwas then applied by searching on the reconstructed imagesforthe planes that containedthe objectswiththe peak andsharpest details to establish the all-in-focus profile. The peakintensityateachpixel alongthelongitudinaldirectionwaschosen and this value became a pixel of an in-focus profile;meanwhile, itslocationinlongitudinal directionwasalsorecorded as a corresponding depth map value. The combina-tion of all-in-focus intensity profiles [Fig. 4(c)] and the cor-respondingdepthmaps[Fig. 4(d)]enabledtoproducethe3-Dvisualizationof microspheresandathresholdontheintensityallowedfordistinguishingobjects fromotherele-ments, which is illustrated in Fig. 4(e). In fact, we took everycenter position of brightest points in Fig. 4(c) as X-Yloca-tionsofparticles,andFig. 4(d)wasusedtodeterminetheZ-location of those particles identified in Fig. 4(c).Figure4(d)ismostlynoisyandZinformationofonlythelocations where the particles were present was actuallyFig. 3Twin-beams DHM setup (reprinted from Ref. 62 by permissionof OSA).Optical Engineering 112306-3 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingused. Figures 4(f) to 4(h) show the XY, XZ, and YZ views ofthe 3-D profile, respectively. This straightforward autofocus-ingalgorithmisabletodeterminefocal planesforall theobjects in the reconstructed volume automatically. The com-binationofdepthmapZand axialX-Ypositionofobjectsallows for quantitative 3-D profiling. This method has beenwidelyusedbyresearchers, andmoreapplicationswill bediscussedin Sec. 4.3.23-DDeconvolution MethodsOverviews of the deconvolution methods regarding the out-of-focus probleminoptical microscopycanbe foundinRefs. 67to70. InRef. 71, 3-Ddeconvolutionmethods,including instant and iterative deconvolution, have beenappliedtorestoreparticledistributionfromhologramsandthecorrespondingreconstructions. Instant 3-Ddeconvolu-tion could be applied on separated particles spread in a cer-tain volume and allowedfortheidentificationofthedepthposition of object and removal of out-of-focus information.Thewaytoperformtheinstant 3-Ddeconvolutionwastoapplyit toreconstructedintensitydistributionsfromholo-grams usingEq. (1):O FT1

FTjUoj2FTjUPj2

; (1)Fig. 4Three-dimensional (3-D) profiling of suspended microspheres. Field of view of (a) to (d) is 90 90m2with464 464 pixels. (a) Hologram. (b) Amplitudeimage. (c) All-in-focusintensityprofile.(d) Depth position profile. (e) 3-D profile (90 90 120 m3) and gray-scale representation of intensity.(f) XY viewof three-dimensional profile (90 90 m2). (g) XZ viewof three-dimensional profile(90 120 m2). (h) YZview of three-dimensional profile (90 120 m2).Optical Engineering 112306-4 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingwhere O is a 3-D object, jUoj2is the reconstructed intensityof the object wave, jUPj2is the reconstructed intensity of apoint scatter [point spread function (PSF)], is a small con-stant toavoiddenominatortobezero, FTis3-DFouriertransform. Asimulatedparticle distributioninthe shapeof letters, , wasplacedat threedifferent planesandeachof themwas constitutedof point scatters, showninFig. 5(a). The reconstructions were performed at continuousplanes in the volume and the PSF was chosen as thereconstruction of the hologram of a point scatter located ata fixed plane. The results of 3-D deconvolution from Eq. (1)showed a sharp 3-D visualization of individual scatters con-stitutingthe three letters, whichwere seentobe distin-guished fromeach other,as shownin Fig.5(b).Iterativedeconvolutioncouldbeappliedonobjects ofcontinuous and extended shapes and distributions. The waytoperformtheiterativedeconvolutionwas toapplyit toreconstructed complex fields from holograms. The iterativeloopincludedO1 UoUno On UPOn1 OnUoUnojUnoj2 n n 1; (2)where Onand Uno are the complex distributions of the objectand reconstructed wave front, respectively. Asimulatedhologramofthreecontinuous,not-point-likeletters, , showninFig. 5(c) was placedat three different planes.Withthesameparametersof reconstructionastheinstant3-D deconvolution case, the iterative deconvolution was per-formed by the iterative loop described above in Eq. (2). Theresult showed the three continuous letters, which were seento be clearly distinguished in a 3-D volume [Fig. 5(d)]. Thesetwo methods were also demonstrated by experimental in-linehologramsof polystyrenemicrospheresrandomlylocatingonbothsidesof athinglass. Acut-out fromtheoriginalhologramof microspheres, whichincludedonlya singlesphere, wastakenasthePSFandthe3-Dlocalizationofmicrospheres was retrieved. Moreover, a simulated PSFwas also tested and a comparable good result was obtained.3.3Rayleigh-Sommerfeld Backpropagation MethodRayleigh-Sommerfeld backpropagation method is a generaland fast volume reconstruction analysis tool for DHM.55,72,73A well-known Gouy phase anomaly55,74can be used to dis-criminate between objectslying oneithersideof theholo-graphic image plane. Arecorded in-line hologramwasreconstructedusingRayleigh-Sommerfelddiffractioninte-gral torefocusthescatteredobject at aparticularpositionand observe the phase anomaly as a contrast inversionabout the objects position by Wilson et.al.72If the complexfield of scattering objects was obtained in the focal plane, itcould be reconstructed at plane z above the focal plane as aconvolution of amplitude in the focal plane with Rayleigh-Sommerfeldpropagator,as in Eq.(3):55Ex; y; z Ex; y; 0 hx; y; zhx; y; z 12zexpikrr; (3)where r x2 y2 z2p. The intensity gradient was thenextractedbasedontheconvolutionwiththeSobel filter,72andifaparticleislocatedinpositivez-direction,thefieldwas converging when it arrived at the image sensor and hadnot yet passed through the geometrical focus. This allowedfor determining the location of the center of the object andseparating objects on different sides of the hologram plane.Particles always had twin images in the reconstructed spaceno matter they are located above or below the focal plane. IfFig. 5(a) and (b) Results of the instant 3-D deconvolution of the intensities reconstructed from the simu-lated hologram of a particle distribution. (c) and (d) Results of the iterative 3-D deconvolution of the com-plex fields reconstructed from the simulated hologram of continuous objects (reprinted from Ref. 71 bypermission of OSA).Optical Engineering 112306-5 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingonly the upstream of the focal plane was reconstructed, theintensitygradientwasamaximumforparticleswithz < 0and a minimum for those with z > 0. A micro-sized diametersphere was tested by this technique and a typical volumetricreconstructed rays converging to the bright point above thefocal planewas showninFig. 6. Another analysis of anexperimental sample consisted of closely spaced particles ona glass slide showing the applicability of this method to dis-tinguish weakly scattering particles, such as colloids or plank-tons locating at close lateral but separate axial positions.3.4Compressive Holographic MethodCompressive sensing is a technique to efficiently acquire theinformationof signalin the underdetermined linear systemsupported by the sparseness of the signal.75Hence, the higherdimensional signal canberecoveredbythelower-dimen-sional measurements. Recently, some research groups havebeen studying about the compressive holography, whichapplied the concept of compressive sensing to hologra-phy.7578Inholographicrecording, 3-Ddataof theobjectwere recorded as a two-dimensional complex hologram.Supported by the condition that the distribution of the objectin 3-D space was sufficiently sparse, the compressive sensingtechnique could retrieve the 3-Ddistribution fromtwo-dimensional complex hologram. Brady et al.75,76hadsuccessfully shown the possibility of the compressiveholographywiththehologramrecordedbyin-lineDHM.In-line DHM,becauseof its simplicity and efficiency, pro-videda potenttoolforcompressive sampling byrecordinga 3-Dobject onto a two-dimensional focal plan array,whichwasrelatedtoFouriertransformoftheobjectfield.Compressive holography collected fewer amounts ofmeasurementsthanvoxelsinthereconstructions. The3-Ddatacube was encoded and compressed into two-dimensionalholographic measurement by holographic sampling process,andtheencodingwastheninvertedusingthecompressivesamplingtheory.Twoseedparachuteswereilluminatedandplacedawayfromthe detector array at different distance separately[Fig. 7]. After reconstruction, the stem and petals represent-ingthehigh-frequencyfeaturesintheimagewereclearlyshownandthedistancebetweenthetwoparachuteswerealsoestimated. Thefinal result presenteda3-Ddatacubeof voxels reconstructed from a single two-dimensional holo-gram,anditdemonstratedtheapplicabilityofcompressiveholography to generate multidimensional images from lower-dimensional data. Adetailed description of compressiveholographytheoryandexperiment resultscanbefoundinRefs. 75and 76.3.5Twin-Beams DHMMethodAs previously depicted in Fig. 3, the twin-beams DHM hadtwo plane beams passing through one objective microscope.Theoretically,thisprocesscouldbesketchedastwoconessuperposedandtransmittedthroughtheparticlesampleP(Fig. 8, left). BeamsB1andB2producedprojectionsP1andP2of particle PontheCCDplane, respectively. Inboth simulation and experiment, the 3-Dcoordinates ofB1andB2pointsandtheradiusof eachbeamcircleonthe CCD plane were first determined based on a simple cal-ibrationmethod.62Then, thecorrespondingcoordinatesofP1 and P2 could be evaluated accordingly from the simplegeometricschematicandthepositionofreal object Pwasdetermined. A real situation of various motile cells floatingFig.6Exampledatafromasingleparticle.Scalebarsrepresent2minall cases.(a)Vertical slicethrough the center of an image stack created by physically translating the sample. (b) Image of a particlelocatedat z 9 m(downstreamof thefocal planeintheilluminationpath). (c)Optical fieldrecon-structedfromthepreviouspanel.Thehologramplane(z 0)wouldbelocatedbelowthebottomoftheimage.(d)Intensity gradient < 0.Thedarkcentral spotisazimuthallysymmetricaboutthezaxisandgivestheparticlelocationinall threedimensions.(e)and(f)Thecompanionimagesto(b)and(c), foraparticlelocatedat z 9 m(upstreamintheilluminationpath). (g)Intensity gradient > 0.Theparticlelocationisspecifiedbyamaximumof intensitygradient forthosescattererswithz < 0(reprinted from Ref. 72 by permission of OSA).Optical Engineering 112306-6 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingFig. 7(a) Raw Gabor hologram for seed parachutes of taraxacum. (b) and (c) Photographs of the indi-vidual objects. (d) Transverse slices at various ranges of the backpropagated (numerical refocusing) field(reprinted from Ref. 75 by permission of OSA).Fig. 8Twin-beams DHM results. Sketch of interference between two beams (a) and estimated path forrandom motion of the three cells (b) (reprinted from Ref. 62 by permission of OSA).Optical Engineering 112306-7 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackinginto a microfluidic chamber was studied using this methodand the 3-D paths of cells were estimated [Fig. 8(b)].62Thephase-contrast maps of the cells were also obtained bynumericallyreconstructingtherecordedholograms.79,804Applications of Three-Dimensional Profiling andTracking by DHM4.1Three-DimensionalProfiling of SuspendedParticlesMicro-sizedparticles, suchaslatexspheresandferromag-neticbeads, suspendedingelatinhavebeenimagedina3-Dvolumebyin-lineDHM.8183Ahologramof objectsandanother backgroundimagewithout theobject presentwere bothrecorded. The difference of these twoimageseliminatedtheartifactsof theilluminationsource. Recon-structionwas achievedbymeans of Kirchhoff-Helmholtztransform84togeneratethecomplexfieldof objects, andthecombinationofastackoftwo-dimensionalreconstruc-tions at various planes resultedinthe3-Dprofileof theobjects. The amplitude obtainedfromcomplexfieldwasusedtorepresent objects. Byinspectingintensityprofilesthroughaparticle, thedepthpositionwasobtainedwherea sharpest intensity appeared. This allowed estimating a par-ticles 3-D position coordinates within submicron. When thederivative of a second-order polynomial fitted to the intensityprofileswastaken, theX, Y, andZpositionsof particlescouldbe determinedwithin 50nm.In Ref. 85, the distribution of dense microparticles(3.2m) suspendedinaliquidvolumewiththedepthofFig. 9Results of 3-D profiling of particles. (a) Part of a recorded hologram using a 10 objective, con-taining 3.189-m-diameter particles in a 1-mm-deep solution. (b) to (d) Reconstruction of planes located120, 580, and 800m from the hologram plane. In-focus particles appear as dark spots on the brightbackground. (e) A combined and/or compressed image containing all the particles covered by the holo-gram section shown in (a). (f) Location of all the particles detected within the entire 1.5 1.5 1 mm3volume, totaling 5769 particles (reprinted from Ref. 85 by permission of OSA).Optical Engineering 112306-8 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and tracking1mmwasmeasured, asshowninFig. 9. Figure9(a)isasection of the recorded hologramand the reconstructedimagesatdifferentdepthsareshowninFigs. 9(b)to9(d).It was clearly observed that the in-focus particlesappearedas dark circular spots. Figure 9(e) shows the combination ofall the reconstructedimages. The peak-searchingmethodwasthenappliedandeachpixel wasassignedthelowestintensity obtained over the entire depth. A thresholding seg-mentationmethodbasedonsignal-to-noiseratio I IIwasappliedonthereconstructedplanes,86whereI isthemeanintensityoftheparticlesinthe volumeandIisthestandarddeviationofintensityoverthisvolume. Alist ofline segments after scanning through the reconstructedimages were combined to two-dimensional planar blobsusing a join operator, which were then united into 3-D par-ticle traces by repeating the peak-searching procedure at dif-ferent depths. The position of a particle was then determinedby using the centroid of the 3-D blob and the distribution ofall theparticlesdetectedwithinthe3-Dvolumewaspre-sented inFig.9(f).4.2Three-DimensionalStudy of MicrofluidicsKreuzers group has studied the microfluidic phenomena byin-lineDHMwiththecapacityoflargedepthoffieldandnumerical reconstruction.87Inafirst experiment, asphereof 150mdiameter was attachedtothe topof a milli-meter-widetank, whichwasfilledwithwater andseededwith small micro-sized latex beads as tracer. The waterwassuckedandpumpedtoflowbyablottingpaper andthe flow past a spherically shaped obstruction was recordedasaseriesof hologramsbyin-lineDHM. Thedifferencebetween consecutive hologram pairs was calculated pixel bypixel (h1 h2; h3 h4; : : : ; hn1 hn) to eliminate back-ground structure and retain only the information of movingbeads, where hn is the nth hologram of recorded frames. Theresulting difference holograms were summed (h2 h1h4 h3: : : h13 h14) into a single hologram, whichwas then reconstructed with Kirchhoff-Helmholtz transformto obtain the focal planes of the streamlines of beads(Fig. 10). The flow information containing sequential posi-tions of beads at successive recording times and the velocityfield defined by vectors connecting two successive positionswas measured and analyzed and the velocity field wasproventobeingoodagreementwithNavier-Stokessolu-tion. In this way, as indicators of flow patterns, the trajecto-ries and velocities of latex microspheres were measured in a3-D volume and time with subsecond and micron resolution.Thesizeand3-Dpositionsofparticlesinflow-throughsystem, suchasfast-movingbubblesinair-water mixtureflows, were measured by Tian et al.88Bubbles were gener-ated by a motor driving a small propeller in a water tank andthedensitywasnot sohighastocorrupt theilluminationbeam. Holograms were first recorded by in-line DHM sys-temandtheFresnel approachwas usedtopropagatetherecordedobject fieldtovarious planes inthevicinityofthe object.55,56At eachpropagationdistance, the Fouriertransformof thehologramwas multipliedbytheFresnelFig. 10(a) 3-Dflowaround a fixed sphere. Ninety holograms were taken at intervals of 0.17 s to generatea difference hologram. (b) One of 60 reconstructions made to render the field velocity shown in (c). In (d)thesolidbluelinesrepresent thesolutiontotheNavierStokesequationforourexperiment andthearrows correspond to the measured velocity field (reprinted from Ref. 87 by permission of Optik).Optical Engineering 112306-9 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingapproximated transfer function and an inverse Fourier trans-form was then applied to obtain the complex field of objectscontaining the intensity profile. A focused bubble appearedtohaveaminimumintensityandthiswasusedasafocuscriteriontolocatebubbles. Atwo-dimensional projectionimage was generated by scanning the reconstructed imagesandrecordingtheminimumintensityvalueforeachpixel.The corresponding depth map where the minimum intensityoccurred was also recorded for analysis. By applying thresh-olding on the minimum intensity projection image, a binaryimageshowingonlytheedgesofbubbleswascreatedandthencombinedwiththedepthmaptodeterminethedepthof objects. The issue of overlapping edges in the projectionimage wasresolvedbyusingaGaussianmixturemodel.89Bubblesizesweredeterminedfromtheareasenclosedbyeach edge. In this way, the 3-D visualization of the positionsandsizesof bubbles waspresented,as showninFig.11.4.3Four-Dimensional Motility Tracking of BiologicalCellsMotility is a major characteristic of certain types of cells, andit is essential to understand the motility of cells by measuringtheir 3-D motion. DHM offers a rapid and efficient method tomonitor the3-Dmotilebehavior anddynamicprocessofvarioustypesofcells. Theeffectivenessofthismethodisquantifiedintermsofthemaximumnumberandspeedofthe moving cells that can be tracked, as well as the directionof the motion (lateral and axial). The results from this workcanbefurther usedtoimplement anoptical trapthat canautomatically track and capture cells with specified charac-teristics,suchas speed,size,or shape.4.3.1Red blood cellsDHMhas been applied in the field of hemodynamicresearch.90A human red blood cells (RBC) sample was sup-pliedtoacircularmicrotubeandtheRBCmotioninthismicrotube flow was recorded sequentially by in-lineDHM. Theangularspectrummethodandquantificationofimage sharpness were applied to reconstruct and locatethefocal planes of cells. Themeasurement accuracies inthe lateral and axial directions were0.3 and1m,respectively. In this way, the full four-dimensional trajecto-ries of RBCs, including space and time information, and thethree-dimensional velocity field were measured, and the lin-earmovement ofcellswasseeninthecircularmicrotube.The feasibilities of in-line DHM, volumetric particletracking, andnumerical reconstructionweredemonstratedwith RBCs.3-D tracking of the sedimentation of RBCs has also beenimplemented by DH-QPMbased on the phase contrastimages.91Thecapturedholograms werereconstructedbynondiffractivespatial-phase-shifting-basedreconstruction92,93toaccesstothecomplexfieldofobjects, whichwasthennumericallypropagatedtodifferent planes byaconvolu-tion-based algorithm.94The phase profile was extractedand phase unwrapping algorithmwas applied for futurefocusingevaluation. Theautofocuscriterionfor asharplyfocusedphaseobject showedminimumvisibility.64,65Thetimedependenceofaxial displacement anddepthpositionwhere the focused phase happened were combined to illus-tratethe3-DprocessofthesedimentationofRBCduetogravitation,as illustratedin Fig.12.4.3.2Free-swimming cellsChilomonas are fast-moving cells, which sense and respondtotheir surroundingsbyswimmingtowardor awayfromstimuli. We have utilized off-axis DHM to track the motilityof chilomonasina3-Dvolumeandtime.21Atime-lapsehologrammovieofthemovement ofchilomonas(averagediameter10m)wasfirstrecordedat30fps. Thefieldofviewwas 90 90m2with464 464pixels. Anexcerptof 14 frames was taken and the difference between consecu-tive hologrampairs was calculated pixel by pixel (h1h2; h3 h4; : : : hn1 hn) to eliminate background structureand retain only the object information, where hnis the nthhologram of selected frames. The resulting seven differenceholograms (from a total of 14) were then summed(h2 h1 h4 h3: : : h13 h14) intoasinglehologram[Fig. 13(a)], which contained all the information on movingcells.Reconstructionbytheangularspectrummethodwasappliedontheresultant hologram, andFig. 13(b)wastheamplitude image reconstructed at the specific planeZ 0. The3-Dtrajectoryofthemovingchilomonaswasbuilt up by first reconstructing the two-dimensional resultanthologramfromZ 80to 120m, in 2-msteps. Theautofocusing method based on peak searching was thenFig. 11Dataprocessingresultsfroma1024 1024 pixel hologram. (a) 3-Dvisualizationof dataprocessing results from a single hologram (diameters not to scale). (b) Bubble size distribution (reprintedfrom Ref. 88 by permission of OSA).Optical Engineering 112306-10 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingemployed, keepingthe maximumintensityvalue at eachpixel in the reconstructed image along Z direction to obtainthe all-in-focus intensity projection [Fig. 13(c)] and combin-ing with the depth position where in-focus intensity occurred[Fig. 13(d)] to determine the focal planes for the entire tra-jectory in the reconstructed volume. A threshold of intensitywasappliedtoeliminatetheunnecessarybackground, andthecenter positionandthemeanintensityof everypointcloud representing every trajectory of chilomonas weredetermined. ThecombinationofdepthpositionZandXYdisplacementofcellsallowedforquantitative3-Dmotilitytracking. The cell was estimated to move a total path lengthof 93 m at the velocity of198ms. Figure 13(e) demon-strates the applicability of DHM for automatic four-dimen-sional tracking of living cells with temporal and spatialresolutionat thesubsecondandmicrolevel. Figures13(f)to 13(h) show the XY, XZ, and YZ views of the 3-D trajec-tory, respectively.21In Ref. 95, a real-time hologram movie of free-swimmingalgae (5 to 10 m in diameter) in seawater was recorded byin-lineDHM. Subtractingtheconsecutiveframesfirst andadding these difference frames then provided a finalimage containingthe time evolutionof algae trajectoriesfree from background interference. A similar peak-searchingreconstruction algorithm as described previously constructedthe 3-D trajectories of algae in seawater approaching a con-centratedsalt solution. Free-swimmingalgaewereseentosense and respond to their surroundings by swimmingtoward or away from stimuli at an average velocityof150ms.Specimenofsmall-sizedbacteria(1500) of human sperm (3to4m)withinalargevolume(>1 L)wastrackedandanalyzed by a dual-viewand dual-color lens-free DHMsetup, which consisted of two partially coherent LEDs of dif-ferent wavelengthsplacedat 45degwithrespect toeachother (Fig. 15).98The recorded in-line holograms only recon-structedatcorrectcombinationofdepth, angle, andwave-length could produce clear images, and the spermheadimages projected at two different wavelengths and viewinganglescouldbeisolatedeveniftheywererecordedinthesame frame. Combining with iterative phase recoverymethod,99the peak-searching algorithm, and related numeri-cal smoothing method, the 3-D trajectories of sperms and thestatistics information of the swim path, pattern, speed, etc.,were revealed (Figs. 15and 16).4.3.4Crawling cells in matrix gelsFerraros grouphas presenteda3-Dtrackingof invitroosteosarcomacellsinagelifiedcollagenmatrixcharacter-ized by a high morphological change.100DH-QPM was uti-lizedtocapturetime-lapsesequencesof cell migrationat12 fph for 15 h to record any movement and morphologicalchange of cells. Numerical reconstruction allowed for evalu-atingbothintensityandphaseprofilesindifferent planes.The cells were first detected from phase profiles by applyinga simple thresholdingfilter, andthis type of pure phaseobjects showed minimumvisibility when they were infocus; therefore, the estimationof focal planes was per-formed by numerical scanning on the stack of reconstructedamplitude images instead for the sharpest details. Then, theaxial position of the center of mass of each cell was deter-mined from phase profiles at the focal plane.101Taking intoaccount the morphological changes of cells, the concept ofminimumboundaryfilters100wasintroducedtodeterminethe positions of cells, and it was compared with otherthreemethods,i.e., centroid,weightedcentroid,andmaxi-mumphase value,as shownin Fig.17.In Ref. 102, to reduce the coherent noise, a partially spa-tial coherent illuminationwasutilizedastheilluminationsource of off-axis DHM setup and an accurate equalizationFig. 14The3-mm-deepsamplevolumeshowingtrajectoriesof P.piscicidabeforeintroductionof prey. Upper inset showsclose-upof part of the sample volume, which is marked by red dash-dot linein the bottomimage [reprinted fromRef. 96, Copyright (2007)National Academy of Sciences, U.S.A].Fig.15Dual-viewlensfree3-Dtrackingof humansperms. (a)Theschematicdiagramof theimagingsystem. Twopartiallycoherentlightsources(redandblueLEDsat625and470nm,respectively)arebutt-coupledtomultimodefibers tosimultaneously illuminatethe sperms at two different angles (red at 0 deg and blue at45 deg). A CMOS sensor chip re-cords the dual-view lensfree holo-grams that encode the position information of each sperm. The 3-Dlocationof eachspermisdeterminedbythecentroidsof itsheadimages reconstructed in the vertical (red) and oblique (blue) channels.(b) The reconstructed 3-Dspermtrajectories. At a frame rate of 92 fps,1575 human sperms were tracked inside a volume of 7.9 L. The timeposition of each track point is encoded by its color (see the color bar)(reprintedfromRef. 98bypermissionof Proceedingsof NationalAcademy of Sciences, U.S.A).Optical Engineering 112306-13 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingbetween the object and reference beams must be reached. Asemiautomatedsoftwarepackageperformedafour-dimen-sional tracking of unstained HT-1080 fibrosarcoma cellsmigration in matrix gels onthe basis of phase profiles,and the 3-D locations of a central point in the cell as a func-tion of time were recorded simultaneously. Quantitative mea-suresonthedisplacement andvelocityof cellswerealsoobtained. By numerical autofocusing and evaluation of quan-titativephaseimages, DH-QPMwasdemonstratedtobealabel-freeandnoninvasivemethodfortrackinglivingcellsin a3-Denvironment.4.4Characterization of MicrofibersKempkes et al. have applied in-line DHM to record a holo-gramofaneedle-shapedtiltedopaquecarbonfiber103andshowed the reconstruction at different planes usingFresnel-Kirchhoff diffractionintegral.94Theintensitypro-files werethenobtainedandconvertedtobinaryimagesby setting a threshold, which retained only the in-focus pix-els. The combination of the stack of binary images produceda 3-Dprofile containingpoint clouds resultingfromthefocusedpixelsofafiberat different reconstructedplanes.A straight line was fit onto the point clouds, and the orien-tation and length of the fiber were estimated. A more com-plexsample of fibers randomlysuspendedinwater wasstudied in the same way. Seven fiber segments were identi-fied and the corresponding orientation and length were mea-sured(Fig. 18). In-lineDHMisshowntobeaneffectivetechnique to characterize nonspherical particles in 3-Dvolume.We have recently studied the nonwoven curved and ran-domly oriented microfibers in suspension.21Fibers werespunontoglasscoverslipsfromco-poly(L-gluatmic acid4,L-tyrosine1)(PLEY)dissolvedinwaterandcrosslinkedasdescribedpreviously.58Thediameter of fiberswas 0.2to5.0m. Figure 19(a) presents a hologramof microfiber.The angular spectrummethodwas appliedtoobtainthereconstructed amplitude image in Fig. 19(b). The microfiberappeared curved and elongated, consistent with our previouswork.104The fieldof viewwas 90 90m2with 464464pixels. The hologramwas then reconstructed fromFig. 16Four major categories of human spermswimming patterns. (a) The typical pattern. (b) The helicalpattern. (c) The hyperactivated pattern. (d) The hyperhelical pattern. The inset in each panel representsthe front viewof the straightened trajectory of the sperm. The arrows indicate the directions of the spermsforward movement. The time position of each track point is encoded by its color (see the color bar). Thehelices shown in (b) and (d) are both right-handed (reprinted from Ref. 98 by permission of Proceedingsof National Academy of Sciences, U.S.A).Optical Engineering 112306-14 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingZ 100to100m, in1-msteps. Addingthese201reconstructed images of amplitude yielded the axial projec-tion shown in Fig. 19(c). The final image had the same num-ber of pixels as eachreconstructedimage. Theprojectedimage was converted to binary by threshold and segmenta-tion, asshowninFig. 19(d). Thebinaryimagewasthenaxiallybackprojectedalongthepathofamplitudeovertheset of reconstructed images. We applied the same autofocus-ing algorithm on the resultant images to record the maximumintensity value at each pixel along the reconstruction direc-tionandtheaxial XYanddepthpositionZof wherethemaximum intensity occurred at the same time. The obtainedall-in-focus intensity profile [Fig. 19(e)] and the correspond-ing depth map [Fig. 19(f)] were then combined to generatethe 3-D visualization of microfiber and a threshold on inten-sitywaschosentohighlight theareaof regional interest.However, there still existed noisy point clouds from specklenoise in the 3-D profile [Fig. 19(g)]. To address this, an aver-ageandpolynomial curvefittingalgorithmaccountingforerrors was adopted. The fitted line of the point clouds (stan-darddeviationsintheXandZdirectionswere0.21and0.06m)inthe3-DcoordinatewaspresentinFig. 19(h).Figures 19(i) to19(k) showtheXY, XZ, andYZviewsofthe3-Dprofile, respectively. Zcoordinateprovidesthedepth information of real image of microfiber in the recon-struction volume. The length of the microfiber in 3-DvolumewasdeterminedfromX, Y, andZcoordinatesofrealimage of microfiber,whichwas126m.214.5CellEnvironment InteractionWhencells swiminthe local environment, theyexhibitmovement toward or away from the stimuli, as well as inter-actionwiththe stimuli. Knowledge of this interactionisessential for understandingthe impact of cells behavioron the process of locating food, avoiding predators, findingmates, etc.4.5.1Three-dimensionalflowfield by swimmingcopepodIn-line DHM was utilized to measure the 3-D trajectory of afree-swimming copepod and the complex flow around it.105A sample of Diaptomus minutus in well water was preparedFig. 173-Dtrackingof invitrocellsinagelifiedcollagenmatrix.Column (a) shows three different cells with the (X, Y) positions esti-mated using the four methods, and the in-focus distance is reportedon the top of each (a). Column (b) shows the trajectories estimated ofthe four methods (reprinted from Ref. 100 by permission of OSA).Fig. 183-D image of a volume of carbon fibers in suspension, viewed from two different directions andshowingboththefittedlines(blacklines), aswell asthepoint clouds(coloreddots)(reprintedfromRef. 103 by permission of OSA).Optical Engineering 112306-15 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingFig. 193-Dprofilingof stationary microfibers. Thefieldof viewof (a) to(f) is 90 90m2with464 464 pixels. (a) Hologram. (b) Amplitude image. (c) Projection image. (d) Binary image, microfiber1 and background 0. (e) All-in-focus intensity profile. (f) Depth position profile. (g) 3-Dprofile(90 90 35m3) andgray-scalerepresentationof intensity. (h) 3-Dfit (90 90 35m3). (i) XYview of 3-D fit (90 90m2). (j) XZview of 3-D fit (90 35m2). (k) YZview of 3-D fit (90 35m2).Fig. 20Left: measured instantaneous velocity near the copepod (a) in the ambient frame of referenceand(b) inthecopepodframeof reference. Right: particlestreaksinthecopepodreferenceframeobtained by combining 130 appropriately shifted reconstructed images. In all cases, the dorsal and lateralviews are in focus (reprinted from Ref. 105 by permission of Journal of Experimental Biology).Optical Engineering 112306-16 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingFig. 213-D displacement of microfiber by swimming paramecium. The field of view of (a) to (h) and (j) is200 200 m2with 768 768 pixels. (a) Hologram 1. (b) Hologram 2. (c) Difference of holograms 1 and2. (d) Amplitude reconstructed at Z 0. (e) Amplitude reconstructed at Z 8 m. (f) All-in-focus inten-sity profile. (g) Depth position profile. (h) All-in-focus intensity of the first track of the displacement of fiber.(i) 3-D fitted line of (h) (200 200 20m3). (j) All-in-focus intensity of the second track of the displace-ment of fiber. (k) 3-D fitted line of (j) (200 200 20m3). (l) 3-D fitted line of the displacement of fiber(200 200 20m3). (m) XYviewof (l) (200 200 m2). (n) XZviewof (l) (200 20m2). (o) YZviewof (l) (200 20m2). For better visualization, the scales of X-Yand Zcoordinate are not in ratio.Optical Engineering 112306-17 November 2014 Vol. 53(11)Yu et al.: Review of digital holographic microscopy for three-dimensional profiling and trackingandseededwith20mpolystyrenespheres. Thein-lineDHMapparatuswithHe-Nelasersourceconsistedoftwoinclined mirrors on the walls of the sample volume to providetwo perpendicular views of the object in the same image. Theseries of holograms that capturedthe copepodwithtwoviews were reconstructed using Fresnel-Huygens princi-ple106with a paraxial approximation to obtain the intensitydistribution. Basedonthelocationof thetracesresultingfromthesuperpositionof threesuccessiveexposures, thecorrespondinglocationof mirroredviews was estimated.Witheachofthetwo views,theparticledisplacementwasmeasured using cross-correlation of the intensity distributionof the three exposures,107,108and the lateral and vertical dis-placements were then combined into a 3-D velocity distribu-tion of particles, as shown in the left panel of Fig. 20. In thereconstructed images sequence movie, the copepod wasseen to sink, swimupward, and then sink again. As itsinks, a recirculating flowpattern was generated due toTable 1 Summary of research progress in the field of digital holographic microscopy (DHM) for three-dimensional profiling and tracking.Object Status of object DHM configuration Method of reconstruction & autofocusing algorithmsMicrospheres/beadsSuspendesusSuspended, sparse Off-axis Angular spectrum sharpness and peak searching21Suspended, sparse In-line Kirchhoff-Helmholtz transform sharpnessand peak searching8183Suspended, sparse In-line Three-dimensional deconvolution71Suspended, sparse In-line Rayleigh-Sommerfeld backpropagation72Suspended, dense In-line Fresnel sharpness and peak searching85Flow-through In-line Kirchhoff-Helmholtz transform sharpnessand peak searching87Seed parachutes Separated position In-line Compressive holographic method75Bubbles Flow-through In-line Fresnel sharpness and peak searching88In vitro cells Floating Twin-beams Twin-beams DHM method62Chilomonas (10m) Free-swimming Off-axis Angular spectrum sharpness and peak searching21Algae (5m) Free-swimming In-line Kirchhoff-Helmholtz transform sharpness andpeak searching95Bacteria (