1

A Copyright Protection Technology

for

3D Printing Model(STL)

Jong (John) CHOI and SK Park(Markany)

Oct 22, 2015

2

1. Clearing Center

2. Feature Extraction

3. Shape Estimation

4. THZ Detector

3

1. Copyright in 3D Printing

Virtuous Cycling for national Industry Eco-system and enhancing

National Competiveness

Capability Building for

Traditional Industry

Clusters (Automobile,

Construction, Manufacturing),

which have been competitive

3D Printing as a means of

Conversion Industry to create

new market and foundation of

national competiveness 3D Innovation in Manufacturing

for converting into service

industry and for Creating value

added competiveness

Automobile

Construction Space, aircraft

3D 프린터산업

의류

medical

3D Printing Industry

4

2. Copyright in 3D Eco System

5

2. Eco Systems for 3D Industry Promotion

standardization certificate

copyright protection system

Fee MGT

system계

quality Certificatecopy prevention

system

product Stability certificate

Industry Control System

Supportive Technology

Development

Industrial

convergence

Outputs

3D scansystem

Markany

copyright protection systems

Fee MGT systems

authoringtools

copy Prevention

systems

인텔코리아 MarkanyETRI

Distribution ToolSystem

construction

medical

artsautomobile fashionmaterial

3D

printing charactersI/O

parts

SW3D printer

Inputs(design)

medical info systems

distribution platform systems

Slicing Technology

6

3. Copyright Clearing Center

Feature Management (FMS)

Meta/feature

database

Main function

Meta/feature scann Fature extraction

Meta/fatureregistration

Meta/faturedeletion

License issue proxy

Proxy/right policy

DB

Right

Policy info

Proxy server

Proxy info

Management Platform

Content MGT Member MGT

Statistic Member

Bulletin MGT

Service platform

Content category

Search/recommend service

SNS

User review Interesting content

Minimum price srvice

Streaming/archiving

storage

Thumb nail

Content/backupstorage

Registration/extraction

Database

Encrypt content

History of Illegal

Distribution

Collectcontent

Violation Monitoring CIMS)

service/met/violation

Monitoring server

functions

Menu Design

Request Feature Recognition

Checklog data

FeaturesRecognition admin

Admin

System MGT

Monitoring

Violation judgement

Illegal distribution of 3D printing filesWeb Hard, Torrent Monitor illegal distribution

Illegal distributor

3Dprinter

Purchaser

Print 3D

DistributionService

Delivery content

Purchase content

Content distribution server (CDS)

Content distribution

Digital Right MGT (DRM)

Packing/license/device

Packaging serverContent

encryptionKey MGT

License server

License issue Log MGT

Integrated with purchase history

Device authentication

Forensic marking Embed Forensic

Mark Extract forensic

mark

Content Management(CMS)

copyright/contract MGT

Content/meta data MGT

Integrate with CP/SP transfer

fee/payment clearing system

Request of content package

RegisterContent

Creator

Database Database

7

4. 3D Printing Process

Forward Engineering

Reverse Engineering

PC

3D Scanner

3D Modeling

CADIAN

Sketchup

Blender

Nurbs

Surface

solid

STL SLICE

Clean Mesh

Fix Mesh

Check volume

Scale

G CODE 3D Print

ControllerProduct

8

5. Issues in Copyright Protection (3D Printing)

(1) Illegal Copy of Model File (STL)

www.shapeway.com (www.****.com)

(2) 3D Scanning and Production

(www.****.com)

9

6. Solutions for Illegal Copy and Legal Dispute

(1) Illegal Copy of 3D Model (STL)

(2) 3D Scanning and Production

(3)Modification of Model File

Encryption (DRM)

Feature(DNA)

Extraction

Shape Estimation

and Matching

THZ Detection

ChungAng Univ. Markany

KAIST

SangMyung Univ.

KEIT

10

7. Encryption of License Storage and STL file

Distribution

legacy

3D Printing Clearance Center

Payment system(PMS)

Content Management system(CMS)

License server

License proxy server CAD Application

Clearing system

DRM server

DRM decryption

Agent

DRM Storage(DRM NAS)

DRM DB

DRMencryption

Storage(NAS)

DB

License file server

purchaser

Copyright management

Systems

AdminWeb

DRMPolicy MGT

11

Seller

UCI

Reg. Date

Distributor

Metadata

(Dublin Core)

Data Size

Padding Size

Signature Base Header Encrypted STL Data Padding

Purchasing STL file

Download

Signature Header Size DRM Header Encrypted Data Padding

Creator

Contributor

Identifier

Publisher

Rights

Title

Subject

Type

Format

Description

.

Signature Header Size DRM Header Encrypted Data Padding

Algorithms: Seed, AES, MACrypto(KCMVP Module), etc Encryption of STL File

8. Encryption of STL Model

12

User Authentication

Content Encryption

License Management

Software Authentication

Hardware Authentication

Intellectual Property Protection

3D model (STL)

user 3D Printer

Purchase of 3D

model (STL)

Application

- Encrypted STL file

- Check whether License

conditions are satisfied

or not

Hardware authentication

Software authentication

Encryption

Issue License

- Check whether SW is legal

- Embed IP information

- Authenticate hardware

- Display/Print IP

information

-micro licensing

- Content encryption (Seed, AES,

KCMVP)

Use 3D Model(STL) Print 3D (STL)

9. STL File Protection

13

1. Clearing Center

2. Feature Extraction

3. Shape Estimation

4. THZ Detector

14

Copyright Protection of

3D STL Model

Certification of

User’s Purchase

History

Monitoring and

Tracing 3D STL

Model

Response to

Collusion Attack

Non-Blind Digital

Forensic for 3D

STL Model

Skeleton Based

Original Model

Estimation

Digital Forensic based on 3D STL

Skeleton with Original Estimation

Digital Forensic Resistant to Collusion Attack

Certification of Purchase History for 3D STL model

Monitoring and Tracing 3D STL Distribution

Extraction of 3D STL model skeleton

Estimation of Original 3D STL Model based on Skeleton

1. Shape Estimation for STL model

15

Cluster Maximization

Feature-Extraction (3D Gradient, Distance,

Surface). Similarity Analysis Algorithm

3D Object’s Feature

Extraction/analysis

ODiSCAlgorithm

Based Clustering

GeneticAlgorithm

Based Clustering

TopologyBased

Feature Extraction

Histogrambased

Feature Extraction

2D View Based

Feature Extraction

1. Feature Extraction and Similarity Measurement

16

2. Skeleton Extraction for STL structure Analysis

• 3D medial axis extraction

original 3D Model (ex) Medial Axis Extraction

17

3D STL modelDefined

skeleton

Extracted

skeletonStructure

Analysis and

Similarity

Measurement

Extraction Module of

3D STL skeleton

Analysis Module of 3D

STL model skeleton

Module for Rapid Extraction and Analysis of 3D STL Model Skeleton

Non-Blind Digital Forensic Algorithm for 3D STL Model

Skeleton Extraction Technology for Topological Structure of

3D STL Model

2. Feature Extraction

18

Estimated

Original

STL Model

of 3D STL

Model

3D STL

Model

Topological

Modification using

Skeleton Structure

Recovery of Surface

based on Skeleton

3 D STL Model Estimation of 3 D STL model Based on Skeleton Structure

Skeleton Structure

DB

Topologically

Distorted/transf

ormed 3D STL

Model

Topologically

corrected 3D

STL Model

3D STL

Model of

Surface

Recovered

Digital Forensic Extraction and Tracing Technology for 3D STL

Model

Estimation of Original Model based on 3D STL Skeleton Structure

3. Feature Extraction and Structure Analysis

19

Feature Extraction/Analysis of STL

Model

Address Length Type Descriptions

0

80

...

...

84

88

92

...

96

100

104

...

...

108

...

...

...

112

80

4

...

...

4

4

4

...

4

4

4

...

...

2

...

...

...

4

ASCII

integer

...

...

real

real

real

...

real

real

real

...

...

integer

...

...

...

real

Header information(not used)

Number of facet(triangles)

First triangle defination

normal vector

x coordinates of normal vector

y coordinates of normal vector

z coordinates of normal vector

first vector

x coordinate

y coordinate

z coordinate

Attribute information

Number of attributes(set to zero)

Second triangle defination

normal vector

x coordinates of normal vector

2D_view based feature extraction

2D view based methods

Histogram based methods

Topology based methods

STL File Format in Binary data

4. Feature Extraction of STL Model

20

Optimization of Features

Feature Extraction/Analysis of 3D Object

Developing Method of Maximizing Homogeneity of Single Unit within Group using ODiSC(Optimized

Discriminative Subspace Clustering) algorithm and Genetic Algorithm

Developing Technology of Extracting Features through Clustering for Pattern Recognition

5. Feature Extraction of STL Model

21

Reverse Analysis of 3D

Model

Feature Extraction DB and optimization

< 2D View DB > < Topology DB> < Histogram DB>

< clustering>

< optimized DB>

6. Feature Description of STL Model

22

3D content

features

features DB

Clustering

Detection result

2D based Feature Extraction After extraction

of each features . Features are combined

Classification of features based on

clustering algorithms

Extraction of topological features

features

3D feature comparsion/analysis

Extraction of Histogram Features

Technology of Similarity based on Features

Extraction of topological features

Extraction of Histogram Features

2D based Feature Extraction

7. Feature Description of STL Model

23

1. Clearing Center

2. Feature Extraction

3. Shape Estimation

4. THZ Detector

24

1. Feature Based Similarity Recognition

Feature Based

Similarity Recognition

Clustering Maximization3D Object Feature Extraction

and Analysis

ODiSC

Based

Clustering

algorithm

Genetic

Based

Clustering

algorithm

Topology

based

Extraction

Histogram

based

Extraction

2D View

based

Extraction

25

Combined Depth Image based Deformed Image Recognition Algorithm

2. Feature Extractions from 3D Model

3D Model(DB Build)

Generate Depth Image

Combine Depth Image

FeatureExtractions

3D ModelNormalization

DeformedImage(Query)

Depth Image Generation

StoreFeatures

FeatureExtraction

FeatureComparison

Output

26

• Normalize using Principal Components Analysis (PCA) , Implicit Shape

Representation (ISR), and weighted implicit Shape Representation(WISR)

technique.

• WISR algorithm can normalize 3D Model so that view point is pointed to sphere

containing the most features

• The depth in 2D images can be invariant, comparing before transformation and after

transformation, when WISR algorithm is applied to.

3. Shape Normalization based on Weighted Implicit Shape Representation

27

• Extract features & Matching: Features of each image can be extracted, and by measuringEuclidean distance between features similarity is evaluated. Then, features of high values are clustered to generate homography.

• Preprocessing : generate Converter Module which will be used in RANSAC sampling process and decide size of grids which 2D image will be segmented into

• Calculate Homography: Homography will be created between Pairs of features generated in “feature extract & match stage” and the image generated in preprocessing stage

• RANSAC algorithm is implemented to create Homography between pairs of features generated in ‘Feature Extraction & Matching’ stage and 2 images generated in Converter Module

(above) matching features,(below) second time feature matching

after complementation

4. Image Preprocessing Based on Homography

28

- Left) Character of Dinosaur model. 3D model with cropped Tail part .- Center) Original 3D Model- Right) Two models are recognized the same model and merged

Case) Deformed image recognition based on combined depth image

5. 3D Model Recognition Algorithm based on Combined Depth Image

29

1. Clearing Center

2. Feature Extraction

3. Shape Estimation

4. THZ Detector

30

3D Pattern

(license)

Inside 3D printed product

Product based license

(3D Pattern layer)

• 3D Printer Resolution : 0.2mm(ave)

• THZ x,y resolution : 0.5mm

• Expected 3D Pattern size : 5 x 5 x 2

• 10 x 10 x 10 bit 125byte

2mm

Scanning: Production Data Embedded

within 3D Printed Products

1. THZ 3D Watermarking: THZ Recognition Technology

31

THz Antenna

3D printed

product

Laser

Degree of Signal transmission

Is different depending on Output Material

THz Imaging Principle

2. THZ 3D Watermarking

32

1. Original Copyright

Holder of 3D Model

2. Date of Printing

3. Printer

(material, type of

printer, IP)

4. Host PC Information

5. Original Right Holder

of Contents

6. Use of Permission

…

1. ㈜samsung P

2. 20150317/11:25

3. ㈜Stratasys, FDP130s

(material, type of

printer, IP)

4. John CHOI

5. ㈜iconix

6. Toys

1. 20byte

2. 10byte

3. 40byte

4. 20byte

5. 10byte…

Min : 125 byte

3D Pattern Layer

2mm

…

3D Pattern Recording

3. THZ 3D Watermarking

33

……THZ data

signal/image processing

3D pattern integration

3D Patternanalysis

…

1. Copyright information of 3D Model

2. Information of Printing Time3. Type of Used Printer

(material, type of printer)4. Data of Host PC5. Use permission

Tomography(single layer image

capturing )

THZ Detector Imaging Process

4. THZ 3D Watermarking

34

TDS Module TDS transponder

equipment

1. Femtosecond laser2. DAQ equipment (Lab view)3. Delay & optics

1. TDS antenna module2. Thz transmitter module3. Object Stage & optics

function THZ Generation & Image processing Thz transmitter & Object reflector part

photo

TDS module case Optical Table

5. THZ Detection Device

35