TECHNO BYTES

Integration of digital dental casts in3-dimensional facial photographsFrits A. Rangel,a Thomas J. J. Maal,b Stefaan J. Bergé,c Olivier J. C. van Vlijmen,a Joanneke M. Plooij,b

Filip Schutyser,b and Anne Marie Kuijpers-Jagtmand

Nijmegen, The Netherlands

Introduction: Since 1915, various researchers have tried to make a 3-dimensional (3D) model of thecomplete face, with the dentition in the anatomically correct position. This was a difficult and time-consumingprocess. With the introduction of 3D digital imaging of the face and dental casts, researchers have regainedinterest in this topic. The purpose of this technical report is to present a feasibility study of the integration ofa digital dental cast into a 3D facial picture. Methods: For the integration, 3 digital data sets wereconstructed: a digital dental cast, a digital 3D photograph of the patient with the teeth visible, and a digital3D photograph of the patient with the teeth in occlusion. By using a special iterated closest point algorithm,these 3 data sets were matched to place them in the correct anatomical position. Results: After matchingthe 3 data sets, we obtained a 3D digital model with the dental cast visible through the transparent pictureof the patient’s face. When the distance between the matched data sets was calculated, an average distanceof 0.35 mm (SD, 0.32 mm) was shown. This means that matching the data sets is acceptable. Conclusions:It seems technically possible to make a data set of a patient’s face with the dentition positioned into this 3Dpicture. Future research needs to establish the value of this 3D fused data set of the face and the dentition

in orthodontic diagnosis and treatment planning. (Am J Orthod Dentofacial Orthop 2008;134:820-6)

Three-dimensional (3D) representation of thedentition in the face has been of interest for along time. In 1915, Van Loon1 introduced his

cubus cranioforus. In this 3D model, he placed plastermodels of the dentition and the face into the anatomi-cally correct positions for the patient. Because of thetime-consuming procedure to make these models, itwas not useful in the orthodontic office.

In the 1980s, 3D images of the maxillofacial regionstarted to develop. These techniques include laserscanning,2 computed-tomography scanning (and ste-reolithography),3 Moiré topography,4 and stereophoto-grammetry.5 All of these techniques have shortcom-ings.2-5 At the moment, stereophotogrammetry seems tobe the most popular technique. With the introduction ofthe 3dMDface (3dMD LLC, Atlanta, Ga) and di3D(Dimensional Imaging, Glasgow, Scotland, United King-

From the 3D Facial Imaging Research Group, Nijmegen and Bruges, andRadboud University Nijmegen Medical Centre, Nijmegen, The Netherlands.aResearcher, Department of Orthodontics and Oral Biology.bResearcher, Department of Oral and Maxillofacial Surgery.cProfessor and chair, Department of Oral and Maxillofacial Surgery.dProfessor and chair, Department of Orthodontics and Oral Biology.Reprint requests to: Anne Marie Kuijpers-Jagtman, Department of Orthodon-tics and Oral Biology, Radboud University Nijmegen Medical Centre, 309Tandheelkunde, PO Box 9101, 6500 HB Nijmegen, The Netherlands; e-mail,orthodontics@dent.umcn.nl.Submitted, August 2007; revised and accepted, November 2007.0889-5406/$34.00Copyright © 2008 by the American Association of Orthodontists.

doi:10.1016/j.ajodo.2007.11.026

820

dom) systems, the applicability of 3D photographs indaily practice became possible.5,6 The capture time forthese images is less than 1 second, and, in contrast toconventional lateral headfilms, no radiation is involved.The limitation of stereophotogrammetry is that it cap-tures only 1 part of the triad bone, soft tissues, anddentition.

Also, new methods for the assessment of the rela-tionship between bone and soft tissue were studied. Thesearch for a substitute for the original cephalogram wasreported in 2 studies. Zhang et al7 compared measure-ments on 2-dimensional facial photographs with ceph-alometric measurements. They concluded that facialphotography is at least as reliable as cephalometrics tostudy facial morphology. In spite of these findings, theystated that lateral headfilms are still necessary for gooddiagnostic purposes, since not all soft-tissue points arein close relationship to the hard-tissue location. In2003, Nagasaki et al8 developed a nonradiographiccephalometric system, in which measurements aremade directly on the patient’s face with a 3D digitizer.This 3D direct cephalometric system has the advantageof no radiation exposure for the patient. In the digitiz-ing process, all landmarks needed for the cephalometricanalysis should be digitized directly on the patient’sface. The software program then constructs a 3Dcoordinate system, with all required points in it. In this

system, it is not possible to add other landmarks later.

American Journal of Orthodontics and Dentofacial OrthopedicsVolume 134, Number 6

Rangel et al 821

No clinically significant differences were found be-tween measurements on this 3D model and lateralheadfilms.

To 3D photographs, a similar coordinate system canbe added; it has the advantage that digital points can beadded whenever the clinician wants.

With the introduction of digital dental models at theend of the 20th century, 3D digital data sets, combiningthe triad bone, soft tissues, and the dentition, haveregained interest.9-11 Curry et al12 developed a 3D dataset, combining 3D facial photographs, digital dentalcasts, and anteroposterior and lateral headfilms. Theonly disadvantage of this method is that 2 x-rays mustbe taken, 1 of which is not routinely taken in orthodon-tic patients—the anteroposterior x-ray.

If it would be possible to add the dentition to the 3Dphotograph, this would give a 3D data set of thepatient’s face with the dentition in an anatomicallycorrect position. This might give enough information tomake lateral headfilms unnecessary for many patients.The purpose of this technical report was to present afeasibility study of the integration of a digital dentalcast into a 3D facial picture to visualize the anatomicposition of the dentition in the face in 3 dimensions.

MATERIAL AND METHODS

For the integration, 3 digital data sets need to beconstructed: a digital dental cast, a digital 3D facialphotograph of the patient with the teeth visible, and adigital 3D facial photograph of the patient with theteeth in occlusion, with the soft tissues as relaxed aspossible.

For the digital dental casts, impressions were madeof the patient with plastic impression trays (TP Orth-odontics, LaPorte, Ind) and orthodontic alginate (CavexOrthotrace, Cavex Holland BV, Haarlem, The Nether-lands). To register the relationship between the arches,a wax bite was made (Tenastyle modeling wax, Kem-dent Associated Dental Products, Wiltshire, UnitedKingdom).

The impressions and wax bite were sent toOrthoproof (Nieuwegein, The Netherlands), where adigital dental cast was constructed. Orthoproof uses aflash computed tomography scanner (Hytec, LosAlamos, NM) to scan the impressions and wax bite.The tube voltage was constant, set at 160 kV, and thevoxel resolution was 0.002 in. The scanner makes780 slices in a rotation of 360°; this takes about 28seconds. Image processing then takes about 20 min-utes for a digital dental cast. By using speciallydesigned software, the digital dental casts of the 2jaws are aligned, with the wax bite as the reference.

When both models are properly aligned, the wax bite

is digitally removed, and the digital dental casts aresent to the orthodontist (Fig 1).

For the digital 3D pictures, a 3D stereophotogram-metrical camera setup and the software program mod-ular system (version 1.0, 3dMDface System) wereused. The camera consists of 2 pods, each containing 1full color and 2 black-and-white cameras. Both podsare connected to a computer (Pentium 4 CPU, 3.2 GHzprocessor speed, 512 Mb RAM, NVIDIA GeForce fx5500 128 Mb graphics card) with a fire wire cable. Thepictures are captured by light photography, withoutradiation. The capture time is approximately 1.5 ms,which limits the risk of artifacts. The 6 simultaneousphotographs are reconstructed into a 3D stereophoto-grammetrical picture (.tsb-file). The digital 3D picturecan now be viewed in the 3dMDpatient softwareprogram (version 2.0). For our study, the .tsb-fileneeded to be converted into an .obj-file, so that it couldbe imported into the Maxilim software program (ver-sion 2.0.3, Medicim NV, Mechelen, Belgium).

For matching the digital dental casts to the digital3D picture, the anterior teeth were used as the matchingsurface. To see the teeth on the digital 3D picture,cheek retractors were used to pull the lips open. Then adigital 3D picture was made, by using the 3D stereo-photogrammetrical camera (Fig 2).

For a good evaluation of the patient’s face, theabove-mentioned picture with cheek retractors is notsuitable. Therefore, a second digital 3D picture was madefrom the patient at rest with the teeth in occlusion (Fig 3).

To integrate the 3 models, the digital dental cast is

Fig 1. Digital dental cast.

matched to the digital 3D picture of the patient’s face

American Journal of Orthodontics and Dentofacial OrthopedicsDecember 2008

822 Rangel et al

with cheek retracters. These 2 models were importedinto the Maxilim software program, which was used formatching (Fig 4, A). The matching was done with theMaxilim 3D texture matching tool. The registrationprocess has 4 steps.

1. Initial positioning of the surfaces. To perform acorrect registration of the digital dental cast and thedigital 3D picture, these surfaces must be aligned.The initial alignment was performed by identifying4 corresponding landmarks spread over the dental

Fig 2. Digital 3D facial photograph of the patient withcheek retractors.

Fig 3. Digital 3D facial photograph of the patient at rest.

surface on both the digital dental cast and the digital

3D picture (Fig 4, B and C). These 4 correspondinglandmarks give an initial registration of the surface.This step minimizes the need for further translationand rotation of the surfaces during the actualregistration (Fig 4, D).

2. Selecting corresponding regions. To improve theaccuracy of the registration, corresponding regionson both data sets were selected. For this study, onlythe maxillary and mandibular central and lateralincisors were selected for registration in both thedigital dental cast and the digital 3D picture (Fig 4,E and F). These parts of the digital 3D picture arethe most reliable regions. The lateral parts becomedistorted on the digital 3D picture. Including thesepoints would reduce the accuracy of the matchingprocedure.

3. Registration of the digital dental cast with thedigital 3D picture. After initial positioning andselecting the corresponding regions, the 3D pictureof the teeth was matched with the digital dental castby using a 3D surface matching algorithm. Thealgorithm used by Maxilim is based on the iteratedclosest point algorithm.13 Specific modifications ofthis algorithm were described by De Groeve et al14

(Fig 4, G).4. Computation of the distance between the digital

dental cast and the digital 3D picture. After per-forming the iterated closest point registration algo-rithm, the differences between the digital dentalcast and the 3D picture were computed. This wasvisualized with a distance kit (Fig 4, H). In thecolored bar of the distance kit, the distance betweenthe 2 surfaces is shown. Each color represents 0.25mm of distance between the 2 surfaces. Greensrepresent the distances when the digital dental castis in front of the 3D picture—the so-called positivedistances—and reds are the distances when thedigital dental cast is behind the 3D picture—theso-called negative distances (ie, dark red means adistance of �2 mm between the surfaces). As canbe seen in Figure 4, H, the distance between thesurfaces on the matched areas was between �1 and�1 mm.

With the digital dental cast positioned into theface, we needed to project a digital 3D picture of thepatient with the teeth in occlusion on the digital 3Dpicture of the patient with cheek retractors. Thisprocedure was the same as the matching between thedigital dental cast and the digital 3D picture of thepatient with cheek retractors (Fig 5). The onlydifference is that the surface matching was done on

the forehead instead of the teeth.

en th

American Journal of Orthodontics and Dentofacial OrthopedicsVolume 134, Number 6

Rangel et al 823

By using the Maxilim software program, it waspossible to show only the digital dental cast and thedigital 3D picture of the patient with the teeth inocclusion. The digital 3D picture of the patient’s facecan be made transparent to see the digital dental cast inthe patient’s face (Fig 6).

RESULTS

The total time for this procedure is about 15minutes for an experienced person.

As shown in the Figures 4 and 5, good matching ispossible for the 3 models. By using the mathematicalenvironment Matlab (R2006a, Mathworks, Natick,

Fig 4. Matching procedure of the teeth: A, malandmarks on the digital dental cast; C, placemcheek retractors; D, matching of the anatommatching (yellow), indicated on the digital deindicated on the 3D facial picture with cheek reH, distance kit, indicating the difference betwe

Mass), it was possible to calculate the average distance

of the 2 surfaces for the matched areas (Figs 4, E and F,and 5, E and F). For this patient, the average distancefor the anterior teeth was 0.35 mm (SD, 0.32 mm). Formatching the 2 digital 3D pictures, the average distancewas 0.29 mm (SD, 0.21 mm).

DISCUSSION

The use of stereophotogrammetry has proven to beuseful in facial and human body 3D imaging.5,15

Although the resolution is high enough for digitizing acomplete face, for parts of the face (eg, the teeth), theimaging is not good enough. After matching the ante-rior teeth of both data sets, some great differences can

lignment of the 2 models; B, placement of thef the landmarks on the 3D facial picture withints (initial alignment); E, region for surfaceast; F, region for surface matching (yellow),rs; G, 2 models after the matching procedure;e 2 surfaces.

nual aent o

ic pontal ctracto

be seen in the distance kit (Fig 4, H). These differences

e kit, i

American Journal of Orthodontics and Dentofacial OrthopedicsDecember 2008

824 Rangel et al

can be ascribed to the resolution of the digital 3D pictures andthe shininess of the teeth and the gingiva. When digital 3Dpictures are made, a digital wire frame (Fig 7) is calculated by

Fig 5. Matching procedure of the 3D pictures:of the landmarks on the 3D facial picture withthe 3D facial picture of the patient at rest; DE, region for surface matching (yellow), indicaF, region for surface matching (yellow), indicatemodels after the matching procedure; H, distanc

the computer. This wire frame consists of many polygons.

From the parts of the face and teeth, which are well visible,small polygons can be constructed. From the parts that areless visible or the shiny surfaces, larger polygons are con-

nual alignment of the 2 models; B, placementretractors; C, placement of the landmarks onching the anatomic points (initial alignment);n the 3D facial picture with cheek retractors;he 3D facial picture of the patient at rest; G, 2ndicating the difference between the 2 surfaces.

A, macheek, matted od on t

structed, resulting in artifacts.

American Journal of Orthodontics and Dentofacial OrthopedicsVolume 134, Number 6

Rangel et al 825

The black parts in the distance kit (Fig 4, H) arewhere the difference between the 2 surfaces is morethan 2 mm. By using Maxilim, it was seen that, duringthe reconstruction of the digital 3D picture, largepolygons were made, resulting in a great differencebetween the 2 surfaces.

When 2 surfaces were matched, the mean distanceand the standard deviation between the 2 surfaces werecalculated. Part of this difference can be caused by thematching error in the program. Our research group isconducting a study to evaluate the measurement errorof the 3dMD camera combined with the Maxilimsoftware program. Even if the distance between thedigital 3D picture with cheek retractors and the digitaldental cast is 0.35 mm, this will probably not beclinically relevant. The last step in the procedure tomake the 3D digital data set is to match the digital 3Dpicture of the patient with the teeth in occlusion withthe digital 3D picture of the patient with cheek retrac-tors. As shown in the Figure 5, E and F, this matchingis performed on the patient’s forehead. We alsomatched with an enlarged matching surface, including

Fig 6. Final 3D data set, with the skin made trfrontal view; B, profile view.

Fig 7. Digital wire frames of the 3D pictures: Athe patient with cheek retractors. The larger pothe teeth.

the supraorbital ridges and the upper part of the orbital

rim. This matching procedure did not give betterresults. From this we can conclude that, for the match-ing procedure, an irregular surface is sufficient. Theinclusion of discrete surfaces does not contribute to abetter matching result.

CONCLUSIONS

It seems to be technically possible to make a 3Ddata set of a patient’s face, with the dentition positionedin the anatomically correct position in this 3D picture.Future research needs to establish the value of this 3Dfused data set of the face and the dentition for orth-odontic diagnosis and treatment planning. When thisproves to be useful, this matching procedure should beautomated by the software manufacturers, so that theorthodontist immediately can start analyzing the dataset.

REFERENCES

1. van Loon JAW. A new method for indicating normal andabnormal relations of the teeth to the facial lines. Dent Cosmos

rent to make the digital dental cast visible: A,

frame of the patient at rest; B, wire frame ofare artefacts caused by the shiny surfaces of

anspa

, wirelygons

1915;57:973-83, 1093-1101, 1229-35.

American Journal of Orthodontics and Dentofacial OrthopedicsDecember 2008

826 Rangel et al

2. McCance AM, Moss JP, Wright WR, Linney AD, James DR. Athree-dimensional soft tissue analysis of 16 skeletal Class IIIpatients following bimaxillary surgery. Br J Oral MaxillofacSurg 1992;30:221-32.

3. Moss JP, Grindrod SR, Linney AD, Arridge SR, James D. Acomputer system for the interactive planning and prediction ofmaxillofacial surgery. Am J Orthod Dentofacial Orthop 1988;94:469-75.

4. Kawano Y. Three dimensional analysis of the face in respect ofzygomatic fractures and evaluation of the surgery with the aid ofMoiré topography. J Craniomaxillofac Surg 1987;15:68-74.

5. Ayoub AF, Wray D, Moos KF, Siebert P, Jin J, Niblett TB, et al.Three-dimensional modeling for modern diagnosis and planningin maxillofacial surgery. Int J Adult Orthod Orthognath Surg1996;11:225-33.

6. Hajeer MY, Mao Z, Millett DT, Ayoub AF, Siebert JP. A newthree-dimensional method of assessing facial volumetric changesafter orthognathic treatment. Cleft Palate Craniofac J 2005;42:113-20.

7. Zhang X, Hans MG, Graham G, Kirchner HL, Redline S.Correlations between cephalometric and facial photographicmeasurements of craniofacial form. Am J Orthod Dentofacial

Orthop 2007;131:67-71.

8. Nagasaka S, Fujimura T, Segoshi K. Development of a non-radiographic cephalometric system. Eur J Orthod 2003;25:77-85.

9. Peluso MJ, Josell SD, Levine SW, Lorei BJ. Digital models: anintroduction. Semin Orthod 2004;10:226-38.

10. Kuroda T, Motohashi N, Tominaga R, Iwata K. Three-dimen-sional dental cast analyzing system using laser scanning. Am JOrthod Dentofacial Orthop 1996;110:365-9.

11. Quimby ML, Vig KW, Rashid RG, Firestone AR. The accuracyand reliability of measurements made on computer-based digitalmodels. Angle Orthod 2004;74:298-303.

12. Curry S, Baumrind S, Carlson S, Beers A, Boyd RL. Integratedthree-dimensional mapping at the Craniofacial Research Instru-mentation Laboratory/University of the Pacific. Semin Orthod2001;7:258-65.

13. Besl PJ, McKay ND. A method for registration of 3D shapes.IEEE Trans Pattern Anal Mach Intell 1992;142:239-56.

14. De Groeve P, Schutyser F, Van Cleynenbreugel JV, Suetens P.Registration of 3D photographs with spiral CT images for softtissue simulation in maxillofacial surgery. Med Image ComputAssist Interv 2001;2008:991-6.

15. Siebert JP, Marshall SJ. Human body 3D imaging by speckletexture projection photogrammetry. Sensor Review 2000;20:

218-26.