6, 2019 mesa, arizona archive - connecting electronic test ... · testconx 2019 test, challenges...

March 3 - 6, 2019

Hilton Phoenix / Mesa Hotel Mesa, Arizona

Archive

COPYRIGHT NOTICEThe presentation(s)/poster(s) in this publication comprise the proceedings of the 2019 TestConX workshop. The content reflects the opinion of the authors and their respective companies. They are reproduced here as they were presented at the 2019 TestConX workshop. This version of the presentation or poster may differ from the version that was distributed in hardcopy & softcopy form at the 2019 TestConXworkshop. The inclusion of the presentations/posters in this publication does not constitute an endorsement by TestConX or the workshop’s sponsors.

There is NO copyright protection claimed on the presentation/poster content by TestConX. However, each presentation/poster is the work of the authors and their respective companies: as such, it is strongly encouraged that any use reflect proper acknowledgement to the appropriate source. Any questions regarding the use of any materials presented should be directed to the author(s) or their companies.

“TestConX” and the TestConX logo are trademarks of TestConX. All rights reserved.

www.testconx.org

Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality. TestConX 2019Keynote

March 3-6, 2019TestConX Workshop www.testconx.org

TestConX 2019

Michael Campbell

Sr. Vice President of Engineering

Qualcomm Technologies Inc. (QTI)

March 2019

Test, challenges with billions of transistors, terabytes of data, 5G mobile and quality.



22

“The world’s most valuable resource is no longer oil, but data”May 6th, 2017 The Economist.

Data is integral to optimizing semiconductorsTest drives immense amount of dataFor QTI and many others, data can be measured in terabytes per month

Yield

Burn in Time to Market

Quality I P Management

Test Time

Actionable data

mjc -testConex

Semiconductor data sources



333

A treasure chest of data, but how you optimize that much data ? Optimized databases and machine learning are some of the keys.

• QTI Semiconductor Landscape

• Major Technology Node or foundry changes every 1 – 2yrs

• 5x more data to analyze on every new technology node (sensors, transistors, conditions, process, etc)

• Requirements:

• Faster time to yield

• Shorter time to root cause yield loss

• Test, Process or Design

• Traditional methodologies do not work well due to complexity and time constraints

• To many transistors

• To many interactive variables

• To much data

Source: Qualcomm Technologies datamjc -testConex



44Source: Qualcomm Technologies data

The amount of transistors has increased over 5x as die size decreased over 50%

In our products, additionally the standard cell and memory bits have increased ~ 3-4x

Technology Migration has Enabled Qualcomm® SnapdragonTM Mobile Platform Capabilities ……..

Source: Qualcomm Technologies data

Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc. and/or its subsidiaries

but with some challenges.

mjc -testConex



55

RF transceiver complexity over time

• Data rates are up year over year - more Uplink + Downlink needed to keep up with data rate needs• Peak Data rates increased ~8x from 2015 thru 2019 while RF complexity increased >1000X.• Result: 5G RF will more 100-1000 x data taken during characterization and potentially manufacturing

4G Gen1

4G Gen2

4G Gen3

5G Gen1

RF Device Complexity 264 12096 207360 1373760Max Data Rate (Gbps) 0.6 1 2 5

0

1

2

3

4

5

6

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1,600,000

Max

Dat

a Ra

te (G

bps)

RF C

ompl

exity

(Car

rier

Aggr

egat

ion

case

s)

DL, UL capability`



66Source: Qualcomm Technologies data

To meet customer demands requires faster time to yield and qualifty !!


Industry demands are driving more capability and faster time to commercialization.


0

30

60

90

120

0

5

10

15

20

25

30

201028nm

2011 2012 201328nm

HP

2014 201520nm

201614nm

201710nm

2018 20197nm

Mas

k Le

vels

Mon

ths

QTI Design to Volume ShipsTime to CS from ES

Time from CS to 1 million units

Time from CS to 10 million units

Mask Levels

mjc -testConex



7

QTI manufacturing process data collection ecosystem

Wafer WAT

Die Finished Good

Assembly TestWafer SortBumpFoundry

In Line Bump Process

B2B File Processing

BumpBE2/

Assembly

Final Test/QA

Wafer Sort SLTWAT

HadoopData validation, file processing and storage

these demandsWorking to build data solutions to meet these demands

When does data get sent to QTI?

SLT

2nd level assemblyl

FP Module

Machine Learning & Advanced Analytics

2nd level assembly

System in a Package

OptimalYield Explorer

Design-centric yield management

PDF Solutions

FT SLT

Fab


Smartest 8Enabling more for

advanced analytics for

digital

mjc -testConex



88

- 200 400 600 800

1,000 1,200 1,400 1,600 1,800 2,000 2,200

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018est

2019est

2020est

2021est

Manufacturing data continues to increase

Increased design/process/test/In Line data is driving data volume.Machine learning increasingly required to detect data shifts.

45nm 28nm

Com

pres

sed

TB

Calendar Years

Cumulative data volume over time

7nm10nm14nm20nm

4x growth

10x growth






9

Inbound AP

I / Loaders

ClusterSQL Engine &

Analytics Library

Application Interface

Distributed MPPDatabase

Recent Summary & Detailed Data

Meta-data

Distributed File SystemHadoop

All Data Query-able FormatA

rchi

ve

Big Data Architecture

Structured Database2

1

3

Inbo

und

Loa

der

Pro

cess

Application Interface

Structured Database

Online Summary & Detailed DataSQL Engine & Application

Server

Client-Server Architecture

Data volume is forcing “Big Data” architectures on analytic applicationsLegacy architectures can’t keep up

Impacts• Data volumes are too large

for a traditional relational database to manage

• Data loading, database & client performance degrade

• Database becomes unmanageable

• Scaling is complex & costly

Results• Migration to big data becomes required to maintain usability

• Increasing • Files sizes.. more tests, more parameters, more die per wafer• Data types.. increased complexity of designs & processes, modules• Analysis.. larger retrievals across operations; cross data correlations

Changes1. Only key meta-data is

stored in structure database

2. Distributed MPP database improves performance & scalability for data analytics

3. Hadoop based batch processing used for larger datasets & timespans

Distributed File SystemHadoop

Raw Data Archive+ ETL to Application

4months of active data ~ 80-100 terabytes of data optimized ~10% meta data; Vertica (distributed db) has ~90% of the data

The new solution – shows 2x faster load , with query performance ~6x - 10x improvement especially on the unit/device level data

mjc -testConex



Machine Learning



11Reference from Optimal Plus.( O+)

Adapt

Learn Act

Validate

Use historical data to build and test a model

If model performance degrades => Refresh the model

Deploy the model and use it to do/decide/alert on something

Measure model performance as things change (all the time)

The Machine Learning Process



12

Tableau

Optimal

Operational Zone Descriptive Reporting

Diagnostic Analytics

Predictive Analytics

Graph Analytics

Machine Learning

Analytics Applications UsersSources

Inline

PCM / WATWafer map

KrigingWS1

WS2

RMA

FABData

BKM

Yield LossDrifting Params

Performance

CustomerData

YieldData

Assembly

FT1FT2

SLTFT3

QA

SATData

ExternalPartners

Vignesh

DataScientists

Land

ing

Stan

dard

izatio

n

Tran

sfor

mat

ions

&Fe

atur

e En

gine

erin

g

Integration

Ingest Prepare Consume

Exploration Zone for Data

Metadata Operational BusinessTechnical

Data Lake Data Warehouse Data Automation

Multiple products

…

Users

AnalyticData Sets TCE PTESD PTE QTI

Cost Opt.

TTR Burn-In

Process

DPPM

Data Models

Similar Business Cases

Common Summaries

Derived Cases

User Data

Best Performing

solution

Qualcomm Analytic Phase 1 statusQualcomm Analytic Phase 1 status

ForecastYield

Performance

SLT

Yield TeamRMA Team

InternalPartners

Version 0.13/20/2018

Self-Service Analytic Tools

Operational Apps

Data Visualization

PythonR

Analytic Applications

QTI Product and Test : Machine Learning System Overview

It’s all about the infrastructure:Need to have the relevant building blocks to create, validate, deploy and monitor ML models in production.The ML code itself is only a small part of the picture.



1313

Capacity limitation

Vertical Ramps

More complicateddesigns in smaller area

DPPM to BPPM

Time to market

Yield

Cost

Quality scale

Production EnvironmentNPI Challenges

$$$

Machine learning can be used to maximize the value of data to deliver yield, quality, and cost.




14

Continuous challenges…

Balancing Quality and Cost of Test:

• Adapt fast to higher production demands w/o quality risk increases.

• Continuous optimization of throughput capacity across family tiers.

• Live KPI monitor to guarantee all levels are met.

• Continue DFT and Test innovation solutions to address new challenges proactively.

New technology nodes require higher number of tests:

• The 0.5% baseline fault coverage gap became TOO BIG to skip.

• EDA tools can’t cover ALL faults in production.

• New fault models are needed• ATE Vector Memory is limited• Higher test times constrain volume

More functional tests @ NPI :

• Covering inter-block defects• Provides gap feedback to DFT• Helps close gap with DPPM• Required with performance tests

Production Ramps are going “Vertical”

• From 9 months to 3 months in latest projects.

• No room for errors• TTR and monitor techniques in place

from day 1.• Identify effective tests ASAP

Smarter and more adaptive test programs are needed!!

Production ramps

# of ATE

050100150200250300350400450500

0100020003000400050006000700080009000

10000

2008 2010 2012 2014 2016 2018 2020

0.5%

Unc

over

ed T

rans

isto

r Gap

(Mill

ions

)

Tran

sist

or C

ount

(Mill

ions

)

Year

Transistor Count vs 0.5% Uncovered Fault Transistor Gap


Mobile industry is now driving toward “almost-automotive” quality levels (<< 100 DPPM)With nearly vertical ramps in new technologies.



151515

A treasure chest of data, using R and python( linked to O+) can create value add data.

People drive decision based on data, ML (Machine Learning) brings value add data!

• QTI semiconductor landscape Using machine learning

• Major Technology Node or foundry changes every 1 – 2yrs

• 5x more data to analyze on every new technology node (sensors, transistors, conditions, process, etc)

• Requirements:

• Faster time to yield

• Shorter time to root cause yield loss

• Test, Process or Design

• Benefits:

• Transforms high dimensionality problems into a simplified version for human analysis

• Self-train / adapts according to the fabrication & test conditions

• Detects systematic patterns related to areas of interest (yield loss, marginalities, HW, etc)




16

Machine learning examples Test time and Quality



17

Critical test List (aka: DO NOT

REMOVE)Quantify Escape Rates & TTRQuantify Escape Rates & TTR

Production:10% Sampling of “Lean” Devices* Used to monitor performance

Production:10% Sampling of “Lean” Devices* Used to monitor performance

Wafer Data Lots

OptimalOptimal

RRAnalytics / ML Algorithms

Identify:•Redundant test•0-DPPM test

Optimize:•Keeps DPPM < 30•Select effective @ defect screen with lowest TT

“LEAN” Test List

Add tests from escapees into “Critical List”

< 30 DPPM?

Continuous check of effective coverage with self-adaptation

NO

YES

XGB Model

PythonPython“LEAN” Test Prog (xx % TTR)

Full Test Program

PerformanceFalse Pass Prediction = 0.10 - 0.39% (Escapes)True Fails Predicted = >75%

Lean coverage with machine learning-Test Optimization and predicting Quality

TTR Savings up to 20% @ < 12 Escapes / mu



18

Forward Looking implementation in production

FabAdaptive

Probe Test

Bump Site

Fabrication Wafer Level

EngineeringData Base

PCM: Parametric Data

Probe: Yield & Parametric

QualModelQual

ModelFab ModelFab Model

AssyModelAssyModelA

ugm

ente

d Te

stsModel

PerformanceMonitoring

Adjust/Re-train

Meets criteria

Fails criteria

FFWD (Feed-Forward)

Advanced Analytics / ML models



19

Quality - Selective devices for Burn-In

Wafer Sort

FT @Temps

Wafer Stage Package Test

Production Parameters

NN

ML

Mod

els

DFFW Risk Class Index

RE-Train criteria from O+

Burn-In

RandomForest

RegressionLogistic

Regression

WAT

WS

FT Hot

FT Cold

FT Room

High Risk? Burn-In

Bypass

To Customer

Goal: Select ONLY the 50% highest failure probability devices for Burn-In stage test while keeping the same DPPM as existing process on record.

yes

no


Read Point

Sample Check

Passing

Passing

Feedback to model retrain

Burn-In Stage

GDBN

DPAT

Samples of Bypass will go into Burn-In to validate passing predictions



20

DFT and fault models

mjc -testConex



214/1/2019

Process Defectivity

Time

NPIProduction

UntestableFaults

UntestableFaults

DFT test coverage gaps: Un-Scan-able (mixed signal, PLL, ..) and coverage lost due to test mode constraints & exceptions

NPIProduction

Immature process with high defectivity

Foreign Material

Non-fill

Line Break

Defect

Design /process margin sensitivities Micro defects leading to model to silicon sensitivities. Circuit sensitivities esp sensitive to process – VT shift, N/P skew

ATE test cost to cover these DFT holes & process defectivity is increased significant esp for Digital part > 5B transistors or ~500M faults

mjc -testConex



22

Testable Gates

Full SOC Logic (including Digital, Analog, MSIP)

~0.1%Logic present in full chip SOC that is not modelled for DFT tools. This could originate from custom logic, analog & mixed signal IPs and certain simplified ATPG models for Qualcomm Proprietary logic gates.

~0.50%A measurable portion of the design that is modelled for the DFT/ATPG tools is deemed untestable due to missing scan structures or test mode constraints, or asynchronous logic, that can prevent ATPG tools to test functional paths.

Logic Modelled for DFT

Logic Testable by SCAN

Faults for ATPG

Basic models: SAF, TDF

99%+detected

faults

Advanced: Cell Aware & others

90%+detected

faults

Aborted Faults

Aborted Faults

DFT & EDA challenges to meet < 100 DPPM requirements

Requires H/W ChangesSource Qualcomm data

mjc -testConex



23

DFT & EDA challenges to meet < 100 DPPM requirements

Test Execution Time

Source Qualcomm data



24

USB on ATE – on chip system test – cost and quality• ATE testing for functional and Scan tests have parallelism constraints due to pins and power for complex

SOC’s. Incorporating a HV SLT could improve throughput and cost due to massive parallelism if enough ATE content can be moved. Test cost could reduce ATE costs while eliminating perceived DFT limitations on ATE due cost reasons. ATPG on SLT could optimize test-cost.

Hypothetical test time breakdown



2525

Summary

Change – it’s a good thing.

• Excel is still a powerful tool, however,

• Machine learning leverages excel, while maximizing the data reviewed and minimizes the time to value add data by > 100X.

• Machine learning DOEs have provided value add in TT, yield, and quality analysis.

• Test over USB is a way to potentially lower COT and overall cost.

• High volume SLT’s may be in the cards for the future

• Automating the mundane machine learning could unleash more engineering creativity

• Lets look at one view of the future. From ITC 2018, author DR Li-C. Wang, and IEA project https://www.youtube.com/watch?v=HpajqoRdz-Q



Nothing in these materials is an offer to sell any of the components or devices referenced herein.

©2019 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.

Qualcomm, Snapdragon, and MSM are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Other products and brand names may be trademarks or registered trademarks of their respective owners.

References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.

Follow us on:For more information, visit us at:www.qualcomm.com & www.qualcomm.com/blog

Thank you

6, 2019 mesa, arizona archive - connecting electronic test ... · testconx 2019 test, challenges...

Documents