Deep Learning ImageAnalysis in Factory Automation

Real-World Applicationsin Production

Olivier DespontCognex Corporation

Example-Based Approach

Adding value –

Best applications for Deep Learning in Machine Vision

When Applying Deep Learning in Machine Vision ?

Self-LearningExample-based training Human-like decisions

Watch dials inspection Cement spots inspection

When Applying Deep Learning in Machine Vision ?

Finds distorted orinconsistent parts

Finds surface defects

Automated bone cutting Rail inspection

Precision Alignment Gauging

Plastic part location Gap check

When Deep Learning is Not Suitable?

What To Look For in a Production Deep Learning System?

Works on commercial PCs

Handles camera mounting effects

Works with limited data sets

Works with high res, color, thermal and 3D images

Image

Informatio

n

Confusio

n Matri

x

Score

Plots

Doesn’t require Ph.D. to configure

∆ η η∑ target - output )( )

What To Look For in a Production Deep Learning System?

Cosmetic InspectionSurface Inspection- Cogwheel Automotive

Cosmetic InspectionSurface Inspection- Cylindrical Motor Mechanism

Rust Spot (OK)

No anomalies (OK) White Area (OK)

Broken (OK)

Defects

Cosmetic InspectionSurface Inspection- Cylindrical Motor Mechanism

and you can also apply this technology

Part LocationDeformable part location and counting

Medical vials counting on a tray

2 images to teach30 min to built the application (incl. training)

Works despite translucent and touching glass vials on shiny metal conveyor with circular background.

Also handles perspective variation due to wide angle lens

Processing time with a GTX 1080 : 80ms/imageImage size : 1800x1450Identification rate : 98%

Bone removal

Part LocationPath following for robot

Pre-insertion screw & debris check

Pre-Assembly VerificationPre-Assembly Obstruction Check

IV bag orientation

Pre-Assembly VerificationPart Correctness and Orientation

Pre-Assembly VerificationKitting & Palletization

Automotive door handles

ClassificationBulk & Batch Product Identification

≠

≠

=

=

Appearance-based diary product identification for logistic application

Image classification with many classesAnd large within class variation

Welding seams overlap (OK)

Correct Welding Seams (OK) Welding seams overlap (OK) Defective Welding Seams (NOK)

Welding seams defect (NOK)

Images courtesy byLEONI Engineering Products & Services, Inc., Lake Orion, MI - USA

ClassificationDefect Classification

ClassificationDefect Classification on Welding Seams

Classify Anomalies based on the results of a region extractor

Images courtesy byLEONI Engineering Products & Services, Inc., Lake Orion, MI - USA

Stamped characters on metal ingotsOCR-A on soft, semi-transparent plastic pouches

Inkjet printed text on bottom of aluminum cans

Characters placed closely to barcode

Molded characters in plastic parts

OCR on non-flat surface with varying light conditions

Pad-printed characters on gas tanksDeformed OCR

Hard-to-Read CharacterDifferent OCR Applications

Post-Placement Packaging Check

Surgical Kit completeness

Post-Placement Packaging Check

Swiss chocolates

What Kind of Applications in Machine Vision Can Deep Learning Solve?

Cosmetic Inspection• Surface Inspection• Functional defect detection

Part Location• Deformable part location and

counting• Path following for robot

Pre-Assembly Verification• Pre-assembly clearance check• Part Correctness and Orientation• Kitting & Palletizing

Classification• Bulk/batch identification• Defect classification

Hard-to-Solve OCR• Distorted character

detection

Post Assembly• Placement check• Final assembly &

packaging verification

human performanc

e and flexibility

Reliabiltyand

consistency

Towards a truly “Smart” Camera …?

What could be the future?

Olivier Despont Product Marketing SpecialistCognex Corp – Switzerland

Olivier.despont@cognex.com

Office +41 26 653 72 78

Contact Information