ssd testing demystified for ssda - · pdf filehdd zone map ssd ftl map fixed zone map map on...

5/2/2013 Copyrighted @ULINK Technology, Inc.

SSD Testing Demystified For SSDA Joseph Chen, ULINK Technology, Inc.

May 2nd, 2013


Agenda

1. SSD Testing Overview

• What is the Difference of HDD and SSD Specification?

• How Do You Test the SSD Specification?

• What do you test for?

2. SSD Test Data Review

• SATA-IO Digital Test Results

• Protocol Test Results

• Regression Test Results

3. FTL Testing • What is FTL Testing?

• How Do You Test FTL?

4. Endurance Testing • The TBW Mystery

• WAF Factor

5. Performance, Really?

• Real World Benchmark?

• TRIM Your Performance?

6. Summary • Magic or Myth?

• Ad Hoc SSD Testing


Using SSD as Primary System Storage

3

Wait!

One More Step…

Coexist and/or Replace HDD

• A huge market is waiting

• On Par w/ HDD Function and Reliability

System Cache Device

• Bridge Between DRAM and external storage

• Compatible with DRAM and HDD function and reliability

Premium Price/Performance Solution

• Can we deliver premiere quality for the price?

• Is an all-electrical components set better?

• What is the MTBF, AFR?


SSD Product Requirement Matrix

Reliability

• MTBF > 1M Hours

• UBER 10-15

• Endurance Rating

Performance

• How fast is the 4KB random IOPS?

• How does the TRIM improve performance?

Functional

• Does the SSD conform to SATA & ATA spec?

• Is the SSD FW and HW designed to spec?

• Does the SSD work on various systems?


HDD & SSD Spec Comparison HDD SSD Comment

General Form Factor 2.5” 2.5" Interface SATA 6Gb/s SATA 6Gb/s Power (Typ/Idle) 1.4W/590mW 6W/650mW NAND Class MLC, 25nm SSD Capacity 500GB/750GB/1TB 100GB/200GB/400GB

RPM 5400 HDD Performance Seq R/W (MB/s) 144 500/460 Random 4K R/W (IOPS) 75K/35K SSD

POR Time (Ave Sec) 8S 2S Cache 8MB HDD

Reliability UBER 10-15 10-15

MTBF (Hours) 1M 2M Endurance TBW SSD

Load/Unload Cycles 60K HDD Function Standard SATA 3.0, ATA-ACS2 SATA 3.0, ATA-ACS2 Features SMART, NCQ, SMART, NCQ, Trim, AES-256 Power Loss Protection Super Cap SSD Environment Shock/Vibration (Gs) 400 1000

Temperature (C) 0-60 0-70 Acoustics (dBA) 24 HDD

Compliance Logo SATA-IO/WHQL SATA-IO/WHQL Agency UL,CE,RoHS UL,CE,RoHS


SSD Testing Overview EVT/DVT RDT

Function Test

Protocol/Command Test Electrical/Power Test EMI/Agency/Logo Test Environmental Test

Reliability Test

Data Compare Test POR/Power Cycle Error Handling Test Mini RDT

MTBF Test Endurance Test UBER Test

Performance Test

IOPS Test MB/s Test Command Service Time Test TRIM Performance Test

Compatibility Test

System Install/Boot Test Application SW Test System Operation Test


SSD Validation Process

DVT (2-4 Weeks)

•Function

•Reliability

•Performance

DVT 2… (2-4 Weeks)

•Function

•Reliability

•Performance

Compatibility Test

•Function

•Reliability

•Performance

Environmental Test

•Electrical

•Environmental

•Agency

RDT (6 Weeks)

•MTBF

•Endurance

•UBER

RTS Ship It!

DVT for SSD Controller Vendors

DVT for SSD Module Manufactures

DVT for OEM SSD Qualification

DVT is the base for all SSD tests


SATA-IO Device Digital Test Summary DUT Sum SDG1 SDG2 SDG3 SDG4 SDG5 SDG6

Errors GTR NCQ ASR SSP IPM DOF H1 0

H2 0

H3 1 1

H4 1 1 H5 0

H6 1 1

H7 0 H8 0

S1 5 1 4

S2 0

S3 1 1

S4 0

S5 0

S6 0

S7 0 S8 0

S9 4 1 2 1

S10 4 1 2 1

S11 0

S12 1 1

S13 0

S14 0

S15 0

S16 0

S17 0

S18 1 1

Test Abb Name SDG1 GTR General Test Requirements SDG2 NCQ Native Command Queuing SDG3 ASR Asynchronous Signal Recovery SDG4 SSP Software Settings Preservation SDG5 IPM SATA Interface Power Management SDG6 DOF Digital Optional Features

1. Total Fail Counts: 19 2. Top Fail Item: NCQ: 6 3. Total Fail Rate: 34.6% (9/26) 4. HDD Fail Rate: 37.5% (3/8) 5. SSD Fail Rate: 33.3% (6/18)

HDD SSD

Good 5 12

Bad 3 6

Ugly 0 0


Protocol Test Summary Test Abb Name PTC1 SEN SecurityEraseNormal PTC2 IDF IdfyInfo_SATA PTC3 SFS SetFeature_SATA PTC4 MAN MandatoryCmds PTC5 WCF WrCacheFlushTime PTC6 RWB RdWrBoundaryCk PTC7 ATA ATACmds PTC8 DID DCO_IdfyInfo PTC9 DCS DCO_SATA PTC10 SMS SmartSet PTC11 SCT SCT PTC12 PWM PwrMgt PTC13 PWS PowerState PTC14 IPM IPM_Cmplt PTC15 SST SSPState PTC16 SSP SSP PTC17 PHY PhyEvntCnt PTC18 DSM ATA_DSM PTC19 NCQ NCQ PTC20 SEC SecuritySet

DUT Sum PCT1 PCT2 PCT3 PCT4 PCT5 PCT6 PCT7 PCT8 PCT9 PCT10 PCT11 PCT12 PCT13 PCT14 PCT15 PCT16 PCT17 PCT18 PCT19 PCT20

Errors SEN IDF SFS MAN WCF RWB ATA DID DCS SMS SCT PWM PWS IPM SST SSP PHY DSM NCQ SEC H1 5 NS 5

H2 0 NS

H3 10 1 3 1 NS 5

H4 10 1 3 1 NS 5 H5 7 3 3 . NS 1

H6 3 . NS

H7 3 NS H8 1 NS

S1 55 6 3 NS NS NS 4 12 NS NS NS NS 6 5 19

S2 22 1 3 3 3 1 11

S3 3 1 2 NS NS

S4 BRK BRK NT

S5 7 2 5

S6 7 2 5

S7 18 NS 3 3 NS 7 5 NS S8 18 NS 3 3 NS 7 5 NS

S9 5 NS NS 5

S10 5 NS NS 5

S11 7 NS NS 2 5

S12 7 NS NS 2 5

S13 2 NS NS 2

S14 2 NS NS 2

S15 13 1 3 7

S16 13 1 3 7

S17 12 1 3 7

S18 5 1 1 1 2

0/26 0/26 0/26 0/26 0/26 1/26 10/26 1/18 2/18 4/25 9/23 3/25 1/25 0/22 0/24 2/24 0/24 7/25 17/24 5/23

1. Total Fail Rate: 14% (64/480) 2. Top Fail Tests: NCQ/SCT/ATA/SEC 3. Average Fail Items: 2.4 (64/26) 4. Average HDD Fail Count: 4.8 (39/8) 5. Average SSD Fail Count: 11.1 (201/18) 6. Broken: 1 SSD

Legend NT Not Tested BRK Broken NS Not Supported

HDD SSD

Good 4 0

Bad 4 17

Ugly 0 1


Regression Test Summary DUT Sum REG1 REG2 REG3 REG4 REG5 REG6 REG7 REG8 REG9 REG10 REG11 REG12 REG13 REG14 REG15

Errors SEN SEE STT DSN SWN RDC RPT RDT PWS IPM DCP MD5 SMT PIT PIW H1 3 1 2 H2 1 1 H3 10059 5 2 H4 11954 5 5 NS 11811 4 121 6 2 H5 10101 2 9 21 H6 5 1 1 2 1 H7 14849 2 7 1 H8 10011 1 1 2 10000 4 3 S1 1 NS NS NS NS 1 NS BRK NT S2 0 2 2 BRK NT S3 1323 NS NS NS BRK S4 0 NT/BRK S5 9 1 9 BRK S6 0 1 BRK NT S7 106 NS NS 1 1 NS 4 100 S8 5 NS NS 1 NS 4 S9 2 1 1 BRK NT

S10 2 1 1 BRK NT S11 2490 1 NS HANG 20 2470 NT BRK S12 0 NS BRK NT S13 10 10 NS BRK NT S14 11 1 10 NS BRK NT S15 10 6 2 3 6 1 NT S16 11 5 2 3 3 1 NT S17 13 4 2 6 1 NT S18 4 4 BRK NT

4/23 4/22 12/23 11/24 2/18 4/23 1/22 2/22 6/22 1/18 8/18 4/17 6/16 6/15 7/13

Test Abb Name REG1 SEN SecurityEraseNormal REG2 SEE SecurityEraseEnhanced REG3 STT SctTemperatureTest REG4 DS1 DstShortOfflineTiming REG4 DS2 DstShortCaptiveTiming REG4 DS3 DstLongOfflineTiming REG4 DS4 DstLongCaptiveTiming REG4 DS5 DstConvOfflineTiming REG4 DS6 DstConvCaptiveTiming REG4 DS7 DstSelOfflineTiming_SmallSpan REG4 DS8 DstSelCaptiveTiming_SmallSpan REG4 DS9 DstSelOfflineTiming_HalfDisk REG4 DS10 DstSelCaptiveTiming_HalfDisk REG5 SW1 SctWrSameTime_PartialBeginDisk REG5 SW2 SctWrSameTime_PartialLastDisk REG5 SW3 SctWrSameTime_WholeDisk REG6 RDC RandomCmds REG7 RPT RandomPIOCmdTime_Distribution REG8 RDT RandomDMACmdTime_Distribution REG9 PWS PowerState REG10 IPM IPM_Cmplt REG11 DCP DataCmp_NCQ_DMA_PIO REG12 MD5 MD5 REG13 SMT SctMaxTemperatureTest REG14 PIT PwrInterrupt REG15 PIW PwrInttrruptWDataComp

1. Total Fail Rate: 26% (78/296) 2. Top Fail Items: PIW/DCP 54%/44% 3. Total Broken Rate: 48% (12/25) 4. Top Broken Items: PWS/PIW/PI/STT 5. HDD Broken Rate: 0% (0/8) 6. SSD Broken Rate: 70% (12/17)

Legend NT Not Tested BRK Broken NS Not Supported HDD SSD

Good 7 2

Bad 1 3

Ugly 0 12


Myth on Reliability Test

Should SSDs and HDDs be tested the same way?

Test Spec HDD SSD

MTBF 1M Hours Mechanical - Spindle

Head/Media

NA

Load/Unload 60K Mechanical - Spindle

Head Stiction

NA

Power Cycle 50K Spindle Motor Ramp/Latch/Park

NA

Endurance TBW NA NAND

Power Interrupt 5K NA FTL


FTL in the SSD FW Block Diagram

Host Cmds

Wear Leveling

NAND

Garbage Collection

Command Decode

Cache/Queue


FTL LBA Mapping Comparison

HDD Zone Map SSD FTL Map

Fixed Zone Map Map on cylinder/head/sector Map created on factory Burn-in

Variable Mapping High degree of variance Complicate meta data management

Static Mapping Map does not change on host command

Dynamic Mapping Map changed on host command

No Update Map does not change on Idle

Continuous Update Mapping Map changed on background tasks (Wear Leveling/Garbage Collection)

Slow Mapping Mechanical seek/latency (~200 IOPS, 5ms)

Fast/Ultra Fast Mapping High IOPS (~50K, 20us) 0.4% background mapping


How to Test FTL

FTL Stability

Stress Test

• Speed Stress

• Access Stress

• S1/S3 Power State Stress

Error Handle Test

• Protocol Error

• Interface Error

Aging Test

• Growing Defects

• Read Disturbance

• Program Disturbance

Disruptive Test

• Power Disruption

• Voltage Disruption

• Asynchronous Events


Myth on Endurance Test Variations of the SSD Endurance Specification Varies? Name Descriptions Workload

Specified

DWPD (Drive Write Per Day)

• Ten full drive write per day, 5 year warranty (Seagate) • Ten drive write per day for 5 years (Intel/Seagate)

No

GBPD (Giga Byte Per Day)

• Minimum of three years of useful life under typical client 3 years workloads with up to 20 GB of host writes per day (Intel)

• 20GB/day of host writes for 5 years under typical client workloads (OCZ)

Typical Client Workload?

TBW (Tera Byte Write)

• TBW: Total bytes written (Micron/Kingston/WD) • TBW: Tera byte write (Sandisk)

No

PBW (Peta Byte Write)

• Lifetime endurance (8KB): Up to 14 PB (Intel) • 4 kilobytes (KB) write endurance of up to 1.5 petabytes (PB) with 20

percent over-provisioning (Intel)

Yes, 4KB/8KB

JEDEC 218/219

• JEDEC TBW: JEDEC 218/219 TBW rating with client/enterprise workload Yes

Note: Information quoted from the public specifications of the list companies for the purpose of illustration


Endurance Test Workload Is 4K/8K Random Write Suitable for Endurance Tests?

Issue: TBW Definition

Not real world workload

TBW is based on WAF, WAF is based Workload

Solution: Use JEDEC Workload for Endurance Tests

𝑻𝑩𝑾 =(𝑺𝑺𝑫 𝑪𝒂𝒑𝒂𝒄𝒊𝒕𝒚 ∗ 𝑵𝑨𝑵𝑫 𝑪𝒚𝒄𝒍𝒆𝒔) ∗ (𝟏 + 𝑶𝑷)

𝟐 ∗ 𝑾𝑨𝑭 𝑾𝒌𝒍𝒅

𝑻𝑩𝑾 = 𝑻𝒆𝒓𝒂 𝑩𝒚𝒕𝒆 𝑾𝒓𝒊𝒕𝒆 𝑬𝒏𝒅𝒖𝒓𝒂𝒏𝒄𝒆 𝑹𝒂𝒕𝒊𝒏𝒈

𝑶𝑷 = 𝑶𝒗𝒆𝒓𝒑𝒓𝒐𝒗𝒊𝒔𝒊𝒐𝒏 = 𝑷𝒉𝒚𝒔𝒊𝒄𝒂𝒍 𝑪𝒂𝒑𝒂𝒄𝒊𝒕𝒚

𝑳𝒐𝒈𝒊𝒄𝒂𝒍 𝑪𝒂𝒑𝒂𝒄𝒊𝒕𝒚− 𝟏

𝑾𝑨𝑭 = 𝑾𝒓𝒊𝒕𝒆 𝑨𝒎𝒑𝒍𝒊𝒇𝒊𝒄𝒂𝒕𝒊𝒐𝒏 𝑭𝒂𝒄𝒕𝒐𝒓 𝟐 = 𝑮𝒖𝒂𝒓𝒅 𝑩𝒂𝒏𝒅 𝒇𝒐𝒓 𝒘𝒆𝒂𝒓 𝒍𝒆𝒗𝒆𝒍𝒊𝒏𝒈 𝒆𝒇𝒇𝒆𝒄𝒕


Myth on Performance Test

Current SSD Benchmarked on 4K Random R/W

SNIA Standardized SSD Performance

Issues – “Unreal”

No real world workload

No TRIM performance

Solution – JEDEC 218 Client Workload

Real world workload

TRIM commands supported


Performance Test Summary

DUT Capacity IOPS

S1 128 GB 1186

S2 256 GB 1879

S3 100 GB 5241

S4 160 GB 2783

S5 80 GB 2442

S6 80 GB 813

S7 120 GB 1632

S8 128 GB 1997

S9 250 GB 1363

S10 250 GB 1050

S11 480 GB 3489

S12 240 GB 2023

S13 120 GB 1856

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5 6 7 8 9 10 11 12 13

IOP

S

DUT

JEDEC Wkld Performance

JEDEC 218A Client Workload IOPS

Tested with JEDEC Client Workload with 38M Commands Issued


Trim Performance Comparison

0

1000

2000

3000

4000

5000

6000

1 2 3 4 5 6 7 8 9 10 11 12 13

IOP

S

DUT

Trim Performance Comparison

No TRIM TRIM

DUT GB IOPS no Trim

IOPS Trim % Diff

S1 128 GB 1186 772 65.09%

S2 256 GB 1879 667 35.50%

S3 100 GB 5241 4615 88.06%

S4 160 GB 2783 3368 121.02%

S5 80 GB 2442 2944 120.56%

S6 80 GB 813 1559 191.76%

S7 120 GB 1632 2430 148.90%

S8 128 GB 1997 2542 127.29%

S9 250 GB 1363 1185 86.94%

S10 250 GB 1050 1132 107.81%

S11 480 GB 3489 2490 71.37%

S12 240 GB 2023 1753 86.65%

S13 120 GB 1856 1644 88.58%



Trim Performance 1 (Improved)

0

1000

2000

3000

4000

5000

6000

7000

1 51 101 151 201 251 301 351

IOP

S

IO Count (in 100K)

Trim Performance Comparison JEDEC 218 Client Wkld

No Trim Snapshot No Trim Average Trim Snapshot Trim Average


Trim IOPS=1559

No Trim IOPS=813

Average IOPS

Instantaneous IOPS


Trim Performance 2 (Reduced)

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

1 51 101 151 201 251 301 351

IOP

S

IO Count (in 100K)

Trim Performance Comparison JEDEC 218 Client Wkld

No Trim Snapshot No Trim Average Trim Snapshot Trim Average


Trim IOPS=2490

No Trim IOPS=3489


Summary

SSD Testing: Magic or Mystery?

Steps to Make SSD to a Premium Product Function

Protocol Test Regression Test for FTL

Reliability JEDEC Workload Endurance Test

Performance Benchmark JEDEC Real-Life Workload IOPS Benchmark TRIM Performance

The Ad Hoc SSD Testing Suites Fills the Gap!


THANK YOU!

ULINK Technology, Inc.

3120 De La Cruz Blvd, Ste 117

Santa Clara, CA 95054

1-408-446-8455

www.ulinktech.com

Contact (at) ulinktech.com

美商宇凌科技有限公司台灣分公司新竹縣竹北市高鐵七路65號16樓之17號

886-3-668-1770

http://www.ulinktech.com/

ssd testing demystified for ssda - · pdf filehdd zone map ssd ftl map fixed zone map map on...

Documents