guided tour from real objects to 3d modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... ·...
TRANSCRIPT
![Page 1: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/1.jpg)
Guided tour from real objects to 3D models
University of DebrecenFaculty of Informatics
HUNGARY
May 2019, Valencia (Spain)
Ildikó Papp
![Page 2: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/2.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 2
Lecture outline• Introduction (Usage of 3D models)
• Model creation based on CT images• CT images
• Segmentation
• 3D scanning• Passive scanning (Photogrammety)
• Active scanning (Structured light scanning, TOF)
• Key takeaways
![Page 3: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/3.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 3
IntroductionUsage of 3D models
![Page 4: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/4.jpg)
3D models• 3D modeling:
Process of developing a mathematical representation of any surface of an object in three dimensions.
• 3D models represent a shape using a collection of points in 3D space, connected by various geometric entities (triangles, lines, curved surfaces, etc.)
• 3D models can be created by designing process or algorithmical methods or scanned.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 4
![Page 5: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/5.jpg)
3D modelsApplications of 3D models:
• Displaying the model in 2D through a process 3D rendering (e.g. computer games or virtual environments)
• Documentation of an object(e.g. architectural or mechanical designs, art works)
• Animated and real-life motion pictures
• Computer simulations of a physical phenomena
• Creating physical things(3D printing, CNC manufacturing or any other manufacturing technology)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 5
![Page 6: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/6.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 7
Model creation based on CT images
![Page 7: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/7.jpg)
CT imagesCT scan = Computed Tomography scan
• Combination of many X-ray measurements and cross-sectional images
• It allows the user to see inside the object without cutting.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 8
CT image series of human brain taken in axial direction frombase of the skull to top.(https://en.wikipedia.org/wiki/CT_scan#/media/File:Computed_tomography_of_human_brain_-_large.png )
![Page 8: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/8.jpg)
3D model from CT imagesMedical image processing software packages:• 3DSlicer
(https://www.slicer.org/ )
• InVesalius(http://www.cti.gov.br/invesalius/ )
• 4Quant Bone Segmenter(https://4quant.com/ )
• Osirix(https://www.osirix-viewer.com/ )
May 2019 TeamSoc21 Workshop 2019 (Valencia) 9
Screens of 3DSlicer with an opened CT image series
![Page 9: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/9.jpg)
3D model from CT images
May 2019 TeamSoc21 Workshop 2019 (Valencia) 10
Screens of 3DSlicer with an opened CT image series
Medical image processing software packages:• 3DSlicer
(https://www.slicer.org/ )
• InVesalius(http://www.cti.gov.br/invesalius/ )
• 4Quant Bone Segmenter(https://4quant.com/ )
• Osirix(https://www.osirix-viewer.com/ )
![Page 10: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/10.jpg)
Result of segmentation
May 2019 TeamSoc21 Workshop 2019 (Valencia) 11
Segmented area of a chest part Generated surface of the same chest part Solid model of the chest part in STL format
![Page 11: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/11.jpg)
Result of segmentation
May 2019 TeamSoc21 Workshop 2019 (Valencia) 12
Solid model of a skull part 3D printed skull part
![Page 12: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/12.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 13
3D scanning
![Page 13: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/13.jpg)
3D scanningA device analyses a real-worldobject or environment to collectdata on its shape.
Collected data:
• points with coordinates
• color (not always).
The collected data can be used toconstruct digital 3D models.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 14
A portable 3D scanner (Sense 3D) in practice
![Page 14: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/14.jpg)
Application fields of 3D scanning• Industrial design
• Quality control
• Reverse engineering
• Prototyping
• Robotics
• Documentation of cultural artifacts
• Display real 3D objects in a VirtualReality
• Healthcare
• …
May 2019 TeamSoc21 Workshop 2019 (Valencia) 15
EinScan Pro 3D Scanner
![Page 15: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/15.jpg)
The results of 3D scanningPrimary result is a point cloud.
• Point could = an unstructured set of 3D point samples (only a collection without any rules, connections)
• A point is represented by its coordinates is a Cartesian coordinate system.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 16
Point cloud of dental arches
![Page 16: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/16.jpg)
The results of 3D scanningSecondary result is polygonal mesh.
• Polygonal mesh creates connection between the collected points.
• Stored in STL and/or OBJ formats.
• STL consists of only triangles.
• OBJ can consist other planar polygonal facets as well. Color data may be associated with it.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 17
Polygonal mesh of dental arches
![Page 17: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/17.jpg)
Efficiency of 3D scanningEfficiency of 3D scanning depends on
• the capacity of the device
• lighting conditions
•
May 2019 TeamSoc21 Workshop 2019 (Valencia) 18
The original scene Surface model with rendering Textured surface model
• brightness of the scanned surface
• user's experience
•
![Page 18: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/18.jpg)
Classification of 3D scanners• Passive technologies:
• mostly based on 2D images
• it doesn't project anything to the object
• e.g. Passive triangulation, Photogrammetry
• Active technologies• Activity means „using some projections”
• Projecting light beams, light line or structured pattern its deformation is investigated.
• Projecting pulsed light waves the time of flight is calculated.
• e.g. Structured pattern projection, TOF (Time of Flight)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 19
![Page 19: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/19.jpg)
Photogrammetry• Based on mix of computer
vision and computational geometry algorithms.
Requirements:
• Images are taken from different viewpoints (at least 20-25 pictures)
• Adjacent images should be overlapping for identification point pairs.
• Advantage: ContrastingMay 2019 TeamSoc21 Workshop 2019 (Valencia) 20
Meshroom (https://alicevision.github.io/#meshroom )
![Page 20: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/20.jpg)
Photogrammetry
May 2019 TeamSoc21 Workshop 2019 (Valencia) 21
http://www.regard3d.org/r3d_demos/demo_sceaux.html Snail’s rude model: https://skfb.ly/6wIR8
Ceramic snail was scanned by SCANN3D. http://scann3d.smartmobilevision.com/
![Page 21: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/21.jpg)
Active scanners (Single line scanning)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 22
• Uses a single line of lasers
• Move it across the surface of the object + take an image in each step
• Time-consuming
• The calculation is based on how the projected line is deformed on the surface.
• Result: high quality surface scan http://wiki.david-3d.com/
![Page 22: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/22.jpg)
Active scanners (Structured light scanning)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 23
Accelerating the process
• Stripped pattern is projected to a surface
• More deformed line can be detected on each picture.
• Different solutions are: lines can be contrast and• monocolor (black-white) or
multicolor
• constant or variable width
Pattern 1
Pattern 2
Pattern 3
![Page 23: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/23.jpg)
Comparison the structured lightning technologies
May 2019 TeamSoc21 Workshop 2019 (Valencia) 24
• single strip• lot of pictures• slow • model with high accuracy• produce very high-quality data
• structured pattern• single picture• fast • model with lower
accuracy
• more strips• more pictures• middle fast/slow• medium accuracy
![Page 24: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/24.jpg)
Post processed scanned modelThe basic post-processing can be performed in the scanner software:
• cutting
• demarcation
• solidifying (patching holes)
More sophisticated steps are required special mesh-processing applications:
• MeshLab (www.meshlab.net/ )
• Meshmixer (www.meshmixer.com/ )
May 2019 TeamSoc21 Workshop 2019 (Valencia) 25
Ceramic snail scanned Sense scanner: https://skfb.ly/6wIRo
![Page 25: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/25.jpg)
The source of idea is in the nature:• The bats (dolphins) use the reflected
sounds to determine location of barriers in front of them or their food. (echolocation)
• For scanning:• Pulsed laser light is emitted
• Reflected light is detected
• The timer measures „time of flight” from starting to return
May 2019 TeamSoc21 Workshop 2019 (Valencia) 26
TOF – Time of Flight
![Page 26: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/26.jpg)
Benefits of TOF• It can capture over 10 000 points in every
seconds.
• High quality scans.
• It’s sometimes hard to tell the difference between photographs and scans.
• Import the scan into a CAD software we can take extremely precise measurements.
• This can help to preserve a particular state of a building or any other built environment.
May 2019 TeamSoc21 Workshop 2019 (Valencia) 27
![Page 27: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/27.jpg)
Key takeaways
28May 2019 TeamSoc21 Workshop 2019 (Valencia)
![Page 28: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/28.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 29
Key takeaways – Technical aspect • Model creation based on CT images
• 3D models are generated by a special software (segmentation)
• The inner part is generated, not only the border surface.
• Used in medical applications, surgical preparation, education
• 3D scanning• Passive technics (no extra lights, patterns are projected)
• Active technics (there are extra lights, patterns during the scanning)
![Page 29: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/29.jpg)
May 2019 TeamSoc21 Workshop 2019 (Valencia) 30
Key takeaways – Societal aspect • Model creation based on CT images
• 3D models help to develop new surgical methods and prepare to the operations
• 3D models required to design unique implants.
• 3D scanning• With 3D scanning the manufacturing processes are more accurate than the
traditional ones.
• 3D scanning preserve the given status of an object. With scans we can recreate things, make copies of an object, can publish things into VR/AR environment.
![Page 30: Guided tour from real objects to 3D modelssociallab.fer.hr/wordpress/wp-content/uploads/2019/... · Guided tour from real objects to 3D models University of Debrecen Faculty of Informatics](https://reader034.vdocuments.site/reader034/viewer/2022050509/5f99f60fb9a20469e22a4318/html5/thumbnails/30.jpg)
Address: Unska 3, HR-10000 Zagreb, CroatiaE-mail: [email protected]: sociallab.education/teamsoc21Facebook: facebook.com/teamsoc21
This document has been prepared for the European Commission however it reflects the views only of the authors, and the
Commission cannot be held responsible for any use which may be made of the information contained therein.
Project reference: 2017-1-HR01-KA203-035408
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.