Computer Science - Research and Development - Special Issue Medical Image Processing - manuscript No.(will be inserted by the editor)

Contact-free volumetric measurement and documentation of facialturgors during the healing period after dental surgery

Christoph John · Ulrich Schwanecke · Dan Brullmann

Received: date / Accepted: date

Abstract In this paper we present a cost-efficient system toempower the contact-free volumetric measurement of facialturgors, see figure 1. Our system enables the reliable mea-surement and documentation of therapy and healing pro-cesses of patients after dental surgeries. At different timeintervals therefor textured depth data are captured with astructured light scanner to subsequently determine volumet-ric models of the turgor under investigation. The volumetricmodels are measured and documented and result in elabo-rate information about a patients anastasis after dental sur-gery. The presented system is currently used in a clinicalstudy to analyze the healing period after third molar removal.

Keywords Structured Light · Volumetric Measurement ·Documentation · Dental Therapy Analysis

CR Subject Classification I.2.10 · I.4.7 · J.3

1 Introduction

The accurate acquisition of three-dimensional characteris-tics of humans or other creatures is becoming increasinglyimportant in biology and medicine. Especially in anthropo-metric investigations, cosmetic surgery, and intensive inves-tigations of the influence of body shape on health risks three-

Christoph JohnDept. of Design, Computer Science and MediaRheinMain University of Applied SciencesE-mail: christoph john@gmx.de

Ulrich SchwaneckeDept. of Design, Computer Science and MediaRheinMain University of Applied SciencesE-mail: ulrich.schwanecke@hs-rm.de

Dan BrullmannUniversity Medical Center of theJohannes Gutenberg-University, MainzE-mail: bruellmd@uni-mainz.de

dimensional information has become indispensable [7]. Re-liable and easy to use systems for surface measurement ofcovering soft tissue are nowadays seen as an inestimably ad-vantage for documentation and therapy planning in differentfields of dentomaxillofacial and reconstructive surgery [14].Based on captured depth data here surface analysis can becarried out to asses the changes in soft tissue aesthetics thatwere induced from orthodondic treatments or soft tissue chan-ges following surgical correction.

A multitude of measurement technologies has been de-veloped in recent years in order to asses three-dimensionalproportions of the human body (see e.g. [16,12]). Contact-less surface measurement methods are thereby most ade-quate for reliable and accurate volumetric measurement ofsoft tissue. The most common contactless technologies usedto measure the human form include depth from focus [11]or defocus [8], moire fringe pattern inference [10], laser-based systems [7] and structured light [4] with white light

Fig. 1: The general system setup. On the right hand side theworking environment of the examiner is depicted. It consistsof the scanner unit composed of a projector and a camera.Both are connected to a computer which runs the measur-ing software. The left hand side of the illustration shows theexamined patient with a facial swelling.

2

projections or infrared light-emitting diodes. These kinds ofcontactless three-dimensional surface scanners also mightbe used more and more in the future to evaluate the geomet-ric fidelity of anatomically shaped tissue engineered con-structs [2], or to digitize gypsum models in dentistry [17].

Surface scanners based on various combinations of oneor multiple cameras as well as light projectors projectingdifferent kinds of light patterns on the specimen seem to bethe most appropriate technology in the medical field, espe-cially because they are fairly harmless to the patient. Thisis due to the fact that in contrast to e.g. laser-based systemsa pattern of nonhazardous visible light with low intensity isprojected onto the object under investigation. Based on theobserved distorted light patterns which are reflected fromthe object’s surface than three-dimensional information canbe recovered.

The just discussed contact-free sensing technologies areirreplaceable for the acquisition of turgors, because any kindof measurement that necessitates contact to the probe wouldunavoidably lead to measuring errors due to the examina-tion. The measuring system described in this article there-fore builds on structured light technology. The system hasbeen developed with emphasis on the quantification of facialswellings prior and after third molar removal. It allows thereliable quantification and documentation of the cheek vol-umes of patients undergoing different medication regimesduring the healing period after dental surgery.

2 Data Acquisition

The presented system is based on structured light technol-ogy in order to realize a cost-efficient and safe device forcapturing of three-dimensional surface structures. Our scan-ner entirely consists of off-the-shelf hardware componentswhich are given as a digital light projector as well as a mono-chromatic USB camera, both supporting XGA resolution.These parts together are sold at retail for less than 1000Euro and hence facilitate the cost-efficient implementationof our system. The scanner captures a working volume ofapproximately 40 × 40 × 40 cm3 at a distance of 110 cmwhich is quite sufficient for measuring of facial swellings.An installation at almost any office workplace or surgery istherefore possible without noteworthy space or cost require-ments. Figure 1 illustrates the general system setup.

The software interface of the presented measurement ap-plication is composed of a MySQL database component,a scan data editor, and different views for scanner control,volumetric measurement, and data analysis. The databasecomponent thereby holds all patient related information likemedical history, surface scans, and registration as well asmeasurement data. This ensures a standardized data accessand moreover allows an easy system extension with external

Fig. 2: The scanner dialog basically allows an examiner tomonitor the position of the patient as well as to control theposition of the black blinder region that protects the patientfrom looking straight into the light of the projector.

software. Both is of major significance for the medical docu-mentation process. All mentioned interface components ex-cept the database are required for volumetric measurementof facial turgors and the analysis of postoperative healing pe-riods. The remaining system components are therefore bestintroduced in the context of a typical volume measurementprocess.

The volumetric measurement of a swollen cheek initiallyrequires the acquisition of a patient’s face surface. This ac-quisition can be done via the integrated capture interface ofthe scanning system which is depicted in figure 2. The cap-ture interface presents a video stream of the scanner camerato allow an appropriate placement of the patient. The scaninterface also loads the scanner calibration file which is pro-vided with the scanner, and the camera configuration param-eters from the most recent scan session. The only parametersthe examiner possibly has to adjust are the height and widthof the black blinder region that is projected onto the patientin order to prevent him from looking into the light of theprojector. A single scan typically requires less than one sec-ond for acquisition. This minimizes measuring errors due tomovements of the patient undergoing investigation.

Once surface data have been measured a manual post-processing step is required to clean the data and extract thepart of the patient’s face that should be investigated. There-fore an editor component was developed which offers easyto use tools for selection and deletion of parts of the mea-sured surface data. Figure 3 depicts an exemplary data setcaptured by our scanning system. Beside the three-dimen-sional surface coordinates of the patient’s face which are

3

measured with an accuracy of about 0.1 mm we also capturecolor information. This color information is especially help-ful in the registration process as described below and allowsthe examiner to empirically assess the turgor. For examplehaematomas can be inspected that way. After data cleaningand extraction of the desired scan regions a polygonal sur-face representation of the remaining point cloud is gener-ated.

In order to evaluate volumetric changes of facial turgorsthe examiner has to capture surface information of the pa-tients cheek. This has to be done several times throughoutthe investigated healing period and thus results in severalthree-dimensional scans of a patients face area of interest. Tosubsequently obtain precise volumetric information aboutthe swellings, the different scan data have to be registeredagainst each other as accurately as possible. This is the cru-cial part of our measuring system, because even small reg-istration errors can propagate to substantial volumetric mea-surement errors of the investigated volume.

Our measurement system implements a two step pair-wise scan registration approach which is subdivided into apre- and a fine-registration stage. The application allows thechoice between two different pre-registration algorithms, anautomatic and a manual one. The automatic algorithm alignsthe barycenters of a scan pair and often results in a sufficientpre-registration for scans with similar topology. This is usu-ally the case if face scans are captured from similar vieworientations. However, if two scans exhibit only small over-lap than the automatic procedure is insufficient and requiresmanual user intervention. A manual pre-registration dialogwas therefore also made available. Here the user needs tospecify four point correspondences between the scans to be

Fig. 3: The scan editor component allows the three-dimensional exploration of the captured surface data. It alsoenables the user to clean up the data and select the facialareas of interest.

Fig. 4: The manual registration dialog provides an easy touse tool to accomplish a coarse pre-registration of a pairof surface scans. The user just has to locate the numberedcrosses at corresponding locations in the texture images ofthe two scans to be registered.

registered (see figure 4). The point correspondences are thanused to compute an initial rigid alignment of the three-di-mensional scan data. Following the pre-registration stagethe roughly aligned scans can serve as input source for amore accurate fine-registration. The presented applicationtherefor implements an “Iterative Closest Point” (ICP) al-gorithm [9,5] which repeatedly finds nearest neighbor pairsbetween scans to iteratively estimate a refined registrationtill the registration accuracy has converged to a local maxi-mum. At each ICP iteration a configurable amount of near-est neighbor pairs (1000 in the default configuration) is ran-domly selected to arrange an overdetermined system of equa-tions which can be solved in closed form [1].

Once a pair of scans was registered they allow for tur-gor volume measurements. A semiautomatic procedure wasdeveloped to find and measure the desired volumes. The ex-aminer is therefor required to manually select a seed pointon an arbitrary surface of the inspected turgor volume. Thisis simply done with a mouse click. The selected seed pointis than used to initialize a region growing procedure [15]which finds the surface shells that clamp the turgor’s vol-ume. Hence, once a seed point was selected, the entire mea-surement process executes automatically. The manual ini-tialization using a seed point metaphor was chosen due to agreat advantage. It allows the assignment of measurementsto a certain measurement series, and with it the parallel main-tenance of multiple series for non-overlapping turgor vol-umes. Such an approach is for example required if severaldistant teeth are removed simultaneously, thus potentiallyresulting in different swelling regions. In such a situation our

4

Fig. 5: Two scans after registration and selection of a seedpoint within the turgor region. A region growing process de-tects and marks the entire area of the swelling.

system can simultaneously document the healing progress ofall the different swellings.

A single face scan consists of approximately 25000 pointmeasurements. Obviously, this denotes that appropriate datastructures have to be chosen to allow interactive responsetimes for seed point selection and also measurement. Theapplication therefore internally represents scans with KD-trees [3] and half-edge data structures [6]. Those allow forfast nearest neighbor search queries between scans, and per-mit the efficient exploration of the local surface neighbor-hood within a single scan. It should be mentioned here thatthe distance between nearest neighbor vertex pairs of differ-ent scans serves as termination criterion for the region grow-ing based turgor search. A result of such a region growingprocess is depicted for an example dataset in figure 5. Thedetected swollen region between a registered pair of scansof a patient’s face is here marked in red.

In the last step of the volumetric measurement processthe swelling, which is given as the enclosed shape betweenthe marked scan regions, is discretized and measured withconfigurable precision. The discretization was implementedas an octree subdivision scheme [13]. This allows the re-alization of very accurate volume measurements within ac-ceptable runtimes. Our system achieves interactive responsetimes even on modesty up-to-date desktop hardware and atthe same time provides an accuracy of about five percent ofthe measured volume. The worst case systematic discretiza-tion error thereby depends on the shape of the measured vol-ume and grows with its surface to volume ratio.

Figure 6 now depicts a typical volume measurement.Here one of the face scans from figure 5 was overlayed witha discretized and measured turgor volume. The intention be-hind this overlay display is to allow for manual user control.

Fig. 6: A surface scan with discretized turgor volume over-lay can be displayed for control purposes.

3 Documentation and Analysis

Once turgor measurements have been computed, they aredisplayed in a table alongside with the patients medical his-tory within the patient summary view. In this view it is alsopossible to select a certain measurement sequence for thepatient and plot the computed measurements over time toget a better impression of the healing process. The patientsummary view is depicted in figure 7.

In the resulting diagram every measurement is plottedas a vertical bar. The center position of the bar at the or-dinate thereby represents the measured turgor volume andthe bar length describes the worst case systematic measure-ment error that was induced from the volume discretiza-tion. The systematic measurement error amounts to approx-imately five percent of the entire turgor volume if a high res-olution discretization was chosen. If required, it would bepossible to further decrease this measurement error throughthe use of a higher resolution discretization. However, inthe presented application this is not necessary as other er-ror sources like registration inaccuracies and the fact that ahuman face is not a rigid object additionally influence the to-tal measurement error. A discretization error of five percentis therefore seen as a sufficiently accurate statistic.

The measurement data which were obtained with oursystem can be exported as comma separated values to allowdata processing in external applications. Thus for examplestatistical software can be employed to perform significancetests. The later is of particular importance if the system isused in larger clinical studies.

5

Fig. 7: The patient summary view of our system shows themedical history together with a selectable measurement se-quence over time. The illustrated example clearly shows de-crease of the swelling during the healing process.

4 Discussion and Future Work

A cost-efficient structured light based system for contact-free volumetric measurement and documentation of facialturgors was presented which has been tailored to the par-ticular demands of dental settings. With a systematic mea-surement error of about five percent this system representsan accurate and reliable measuring tool. It is currently usedin a clinical study at the University Medical Center of theJohannes Gutenberg-University in Mainz. In this study, thetotal accuracy of our system and the technical relevance ofthe volumetric measurements for quantification of healingprocesses will be investigated further.

The next step to improve our system would be a com-pletely automatic registration process which makes the pre-registration stage obsolete. This is of particular importancebecause the pre-registration step requires input of an userand is therewith cumbersome and error-prone. A completelyautomatic registration procedure will further increase the re-liability and usability of our system. To realize an automaticregistration we will incorporate the fact that we always makescans of human faces. Thus additional knowledge of surfacetexture and curvature can be employed for registration. Be-sides an automatic registration we also intend to provide ad-ditional metric measurement tools for surface area and dis-tance computation as those represent complementary infor-mation queues for the quantification of healing processes.

Acknowledgements We would like to thank PD Dr. med. dent. RalfSchulze from the Dept. of Oral Surgery (and Oral Radiology) at theUniversity Medical Center of the Johannes Gutenberg-University inMainz for a lot of extensive discussions and comprehensive insightsinto dentistry. Part of this work was funded by the PRO INNO II pro-

gram of the German Federal Ministry of Economy and Technology(BWMi).

References

1. K. S. Arun, T. S. Huang, and S. D. Blostein. Least-squares fittingof two 3-d point sets. IEEE Trans. Pattern Anal. Mach. Intell.,9(5):698–700, 1987.

2. JJ. Ballyns, D. Cohen, E. Malone, SA. Maher, HG. Potter, TM.Wright, H. Lipson, and LJ. Bonassar. An Optical Method forEvaluation of Geometric Fidelity for Anatomically Shaped Tis-sue Engineered Constructs. Tissue Eng Part C Methods., Sep 29,2009. Electronic publication ahead of print.

3. J.L. Bentley. Multidimensional binary search trees used for asso-ciative searching. Communications of the ACM, 18(9):509–517,1975.

4. P.J. Besl. Active, optical range imaging sensors. Machine visionand applications, 1(2):127–152, 1988.

5. P.J. Besl and H.D. McKay. A method for registration of 3-Dshapes. IEEE Transactions on Pattern Analysis and Machine In-telligence, 14(2):239–256, 1992.

6. M. Botsch, S. Steinberg, S. Bischoff, and L. Kobbelt. Openmesh-a generic and efficient polygon mesh data structure. In OpenSGSymposium, 2002.

7. T. Bretschneider, U. Koop, V. Schreiner, H. Wenck, and S. Jaspers.Validation of the body scanner as a measuring tool for a rapidquantification of body shape. Skin Research and Technology.

8. S. Chaudhuri and A.N. Rajagopalan. Depth from defocus: a realaperture imaging approach. Springer Verlag, 1999.

9. Y. Chen and G. Medioni. Object modeling by registration of mul-tiple range images. Image and Vision Computing, 10(3):145–155,1992.

10. K. Creath and JC Wyant. Moire and fringe projection techniques.Optical Shop Testing, pages 653–683, 1992.

11. J. Ens and P. Lawrence. An investigation of methods for determin-ing depth from focus. IEEE Transactions on Pattern Analysis andMachine Intelligence, pages 97–108, 1993.

12. M. Josten, D. Rutschmann, and R. Massen. Messbar einfach:Mobiles und wirtschaftliches 3D Body Scanning in der Medizinmit dem MagicalSkin Scanner(TM). In Bildverarbeitung fur dieMedizin, pages 216–219. Springer, 2003.

13. ME Mehl and DJ Meagher. Geometric Modeling Using Octree-Encoding. Computer Garphics and Image Processing, 19(2):129–147, 1982.

14. K. Schwenzer-Zimmerer, BI. Boerner, NF. Schwenzer, AA.Mueller, P. Juergens, A. Ringenbach, E. Schkommodau, and HF.Zeilhofer. Facial acquisition by dynamic optical tracked laserimaging: a new approach. Journal of plastic, reconstructive & aes-thetic surgery: JPRAS, Sep. 62(9):1181–1186, 2009. Epub 2008Jul 3.

15. L. Shapiro and G.C. Stockman. Computer Vision, 2001.16. N. Stein and B. Minge. VIRO 3D: fast three-dimensional full-

body scanning for humans and other living objects. In Proceedingsof SPIE, volume 3313, page 60. SPIE, 1998.

17. J.M. van der Zel. Implant planning and placement using opticalscanning and cone beam CT technology. Journal of Prosthodon-tics, 17(6):476–481, 2008. Electronic publication 2008 May 9.