pci-express technology update and compliance testing 5 amf 2010 pci-sig pci express standards...
TRANSCRIPT
PCI Express 3.0
Portions Copyright 2010 PCI-SIG
PCI-Express Technology
Update and Compliance Testing
AMF 2010Page 2
Agenda
Introduction to PCIe 3.0 8GT/s
Physical Layer Challenges at 8GT/s
Protocol Testing Key Considerations
AMF 2010Page 3
Agilent Digital Standards Program
• Our solutions are driven and supported by Agilent experts
involved in international standards committees:
– Joint Electronic Devices Engineering Council (JEDEC)
– PCI Special Interest Group (PCI-SIG®)
– Video Electronics Standards Association (VESA)
– Serial ATA International Organization (SATA-IO)
– USB-Implementers Forum (USB-IF)
– Mobile Industry Processor Interface (MIPI) Alliance
– And many others
• We’re active in standards meetings, workshops, plugfests, and
seminars
• We get involved so you benefit with the right solutions when you need
them
SuperSpeed USB
AMF 2010Page 4
We understand your future requirements,
because we help shape them
Jim Choate
USB-IF Compliance Committee
USB 3.0 Electrical Test Spec WG
WiMedia CRB
Rick Eads
PCI-Sig Board
Member
Brian FetzDisplayPort Phy CTS Editor
VESA Board Member
Min-Jie Chong
SATA 6G / PHY / LOGO Contributor
SATA-IO Gold Suite Lead
The Agilent team maintains engagement in the top high
tech standards organizations
Perry Keller
JEDEC Board Member
AMF 2010Page 5
PCI-SIG PCI Express Standards Organization
PCI Express Board of DirectorsAgilent, Intel, IBM, LSI Logic, Dell, HP, Oracle (Sun Microsystems), nVidia, AMD
PCI-SIG Executive Director: Reen Presnel, VTM
Electrical
Work Group:
Intel, AMD
Protocol
Work Group:
Intel
Card
Electromech
anical
Work Group
Cable work
group
Legal: Tim Haslach
PCI Express 3.0
Serial
Enabling
Work Group
Electrical Spec
Protocol Spec
C.E.M Spec
Cable Spec
Test Specification
AMF 2010Page 6
PCI Express Timeline
1999 2001 2003 2005 2007 2009 2011 2013
20
40
60
80
GB
/s
PCI/PCI-X
PCIe Gen1 @ 2.5GT/s
PCIe Gen2 @ 5GT/s
PCIe Gen3 @ 8GT/s
Atomic Ops, Caching hints
Lower latencies
Enhanced Software Model
IO Viritualization
AMF 2010Page 7
Agenda
Introduction to PCIe 3.0 8GT/s
Physical Layer Challenges at 8GT/s
- Transmitter (TX)
- Receiver (RX)
- Channel
Protocol Testing Key Considerations
AMF 2010Page 8
Physical Layer Differences at 8GT/s
• Data rate for PCIe Gen3 is 8GT/s
• 8B/10B encoding vs. scrambling
• Reference point for voltage and jitter analysis of transmitters
• Compensation for Test Fixture Losses. The use of de-
embedding.
• Transmitter based equalization (de-emphasis)
• The role of receiver equalization
• The need to established clear channel performance
requirements.
AMF 2010Page 9
Transmitter Measurements
Replica S-parameters are
measured and channel
inverse applied to
measurements at TP1
DUT
Low jitter
clock source
TP1TP1
DUT
Breakout
ChannelLow jitter
clock source
TP3TP2
Replica
Channel
Length 3 – 6”
A replica channel is
required to permit de-
embedding to device pin
AMF 2010Page 10
Your Goal: Develop a System Model
DieBonding
wire/pins
Pc
transmission
line
Standard
connector
Pc
transmission
line
Standard
connector
Bonding
wire/pinsDie
Starts in here Ends in here
1) Measure at a point you
can probe
2) Get accurate answers where
probing is not possible
Tx
Die
Bonding
wire/pins
Pc
transmission
line
Standard
connector
Pc
transmission
line
Standard
connector
Receiver
Die Pad
Transmitter Receiver
AMF 2010Page 11
Signal Path Flow for De-Embedding
EQ+
-
Connecto
r
TP0 TP1
Channel
Con
ne
cto
r
EQ+
-
TP2 TP3 TP4
Txp
Txn Rxn
Rxp
Tx Rx
Signal generated here
Exits IC here
Exits board here
Combine measurements and transmission line
models to view simulated scope measurements
at any location in your design
Load S-Parameters into Signal Path
AMF 2010Page 12
S-Parameters
S2P File
S21 Insertion Loss
AMF 2010Page 13
Transmitter Based Equalization (De-emphasis)
• 2.5 GT/s: Same De-emphasis for all
• 5GT/s: Introduced platform dependent options selectable
on a per-link basis
– -3.5dB (client) -6dB (server)
• At 8GT/s, a static selection for de-emphasis is problematic
• Need additional levels to compensate for the inverse of
step response for target channels
• May need to provide for finer grain control to maximize eye
opening.
AMF 2010Page 14
Transmitter Parameters at 8GT/s
• Receiver equalizer changes the game vs. earlier
generations
• Jitter budget to include unrecoverable jitter
• Data dependant jitter substantially recoverable with equalization
• Jitter that is uncorrelated to the data comprises a larger portion of jitter
budget.
• The effect of jitter amplification of a channel
• AC Voltage Measurements impacted by skin effect losses
• How to compensate?
Channel
Jitter Amplification
AMF 2010Page 15
Sample Measurement of Actual PCIe 3.0 Silicon
Coax cables to
scope
TP1
SMA connector
SMA
Rogers PCB
PHY chip
~5cm
81134A
100MHz, square, 0-200mV
Tr = 51psTypical rise times for PCIe 3.0 Silicon
observed between 50-66ps
This is typical for process nodes of 45-
65nm.
AMF 2010Page 16
Magnitude of Harmonics
Tr = 42ps
Rise time of pulse generator measured on
30GHz Sampling Oscilloscope
26.5GHz Spectrum Analyzer measurement of
Pulse Generator output at 8GT/s, 600mV p-p
AMF 2010Page 17
Comparison of Oscilloscope Noise Floor (in dBm)
to Magnitude of 8GT/s Harmonics
At 100mV/div,VRMS=2.97mV= -37.5dBm
Noise floor of 12.5G
Scope
Magnitude of Harmonics of 8GT/s Signal
AMF 2010Page 18
Minimum Scope BW for PCIe 3.0
• Recommendation Submitted: Add the following text as
clarification text to 4.4.1.3.1 (Data Dependent and
Uncorrelated Jitter) or to the notes below table 4-29.
• Benefits:
• Provides clear direction to what is today an implied requirement.
• HF artifacts impacting de-embedding accuracy reduced.
• Current investment in PCIe 2.0 TX test equipment preserved for
PCIe 3.0 development.
Section 4.3.3.5, table 4-17, note 2 states, ‘measurements at
5.0 GT/s require an oscilloscope with a bandwidth of ≥12.5
GHz, or equivalent.’ This requirement also extends to
measurements at 8GT/s”.
AMF 2010
PCI Express Process Nodes
Page 19
Year Process size
1999 180 nm
2001 130 nm
2003 90 nm
2005 65 nm
2007 45 nm
2009 32 nm
2011 22 nm
2013 15 nm
2015 11 nm
PCIe 3.0 target process node
PCIe 3.0 target observed process
node
The smaller the process size
the faster the edge rate achievable
AMF 2010
Edge Rate Drive Bandwidth Needs
Page 20
32GHz –12.6ps Edge 16 GHz – 33ps Edge
AMF 2010
DSA-90000-X Series Noise Performance
Page 21
LeCroy Frequency
Interleaving
Frequency
Interleaving
DSP
Boosting
32GHz Noise floor
@100mV/div
=3.923mVRMS
=-35.12dBm
AMF 2010
Low Noise+High BW=More Harmonic Content
Page 22
Content below Oscilloscope Noise Floor @ 32GHz
Harmonics of
a 12.6ps Edge
@32GHz
AMF 2010
More BW = More Margin for next gen process nodes
Page 23
Eye at 20GHz BW Eye at 32GHz BW
AMF 2010Page 24
Receiver Testing at 8GT/s
• Jitter Stress a
combination of SJ, RJ,
ISI, CM/DM noise.
• Specification requires
performance to
compensate for breakout
channel losses.
• Multiple calibrations
channels accommodate
differences in ecosystem
channels
breakout
channel
replica
channel
Test Board
TP2
8G/s
Pattern
generator
TP3
BER
TP1
Signal
generator
CombinerCombiner
Calibration
channel #1
Calibration
channel #2 RefclkRx DUT
Apply behavioral
Rx Eq and CDR
Stressed
eye
TP4 TP5
Rx Testboard Topology
Gen3 DUTAgilent Technologies
1.5 2.0 2.5 3.0 3.51.0 4.0
20
15
10
5
25
0
S21
(dB
)
Frequency (GHz)
-22 dB
-12 dB
-3 dB
breakout + long
calib channel
breakout + short
calib channel
breakout
channel only
1.5 2.0 2.5 3.0 3.51.0 4.0
20
15
10
5
25
0
S21
(dB
)
Frequency (GHz)
-22 dB
-12 dB
-3 dB
breakout + long
calib channel
breakout + short
calib channel
breakout
channel only
Sample Insertion loss guidelines for calibration channels
AMF 2010Page 25
Channel Analysis
AMF 2010Page 26
Title text – Arial 28 ptSub-title – Arial 24 pt Regular
Introduction to PCIe 3.0 8GT/s
Physical Layer Challenges at 8GT/s
Protocol Testing Key Considerations
AMF 2010Page 27
Protocol Layer Changes and Challenges
• Probing challenges due to the physical layer
– Close eye spec, need mechanism to recover signal
– How to probe (view) the signal without breaking the link
– How to probe (view) the signal without changing the actual signal
• Protocol encoding & encapsulations changes
– New encoding mechanism 128/130 with sync header
– Scrambling only (no more 8B/10B encoding)
– No K-Codes for framing and link management
• LTSSM changes, and lack of devices for testing
– Handle additional speed change permutations
– Change in the state machine to handle equalization
– No (limited) devices on the market to test against
AMF 2010Page 28
Protocol Layer Changes and Challenges
•
• Protocol encoding & encapsulations changes
• LTSSM changes, and lack of devices for testing
AMF 2010Page 29
PCI Express 3.0 – Probing Challenges
PCIe 1.1, 2.5 GT/s
16” Channel
PCIe 2.0, 5.0 GT/s
16” Channel
PCIe 3.0, 8.0 GT/s
16” Channel
AMF 2010Page 30
PCI Express 3.0 - Probing Challenges
Probe Point ?
1 2
Device BDevice A
RX
RXTX
TX
Need to compensate
for the de-emphasis
from the transmitter
Need to compensate for the
channel loss, need similar
equalization as receiver
Probe Point ? Probe Point ?
2
Need to compensate
for unsettled reflections
AMF 2010Page 31
Validating Probe Behavior
Test Board
Probe
+
Equalizing
Component
(ESP)
Use J-BERT to inject
interference into the test
board (test pattern #5)
Use the PCI-SIG defined
Client channel models
Measurement of before and after signals
Agilent N4903B J-BERT
AMF 2010Page 32
PCI Express 3.0 – ESP Results
Agilent ESP (Equalization Snoop Probe):
– Supper receiver to account for a wide spectrum of losses (probing at
different points on the bus)
– Auto tuning, to account for being plugged into any location in the channel
– Compensation for probe cable losses
Unequalized Signal at 8GT/s Equalized Signal at 8GT/s
Protocol Analyzer Probe with
Custom Tunable Equalization
(ESP)
AMF 2010Page 33
Agilent Probes – with ESP Technology
Slot Interposer:
– All link widths (x1, x4, x8,
x16) supported
Mid-bus probes:
– Footprints are available now
– KOV available now
– Single probe supports up to x8,
two together for x16 support
AMF 2010Page 34
Protocol Layer Changes and Challenges
• Probing challenges due to the physical layer
•
• LTSSM changes, and lack of devices for testing
AMF 2010Page 35
Protocol Encoding Changes
PCIe Gen1 & Gen2 Encoding
– 8B/10B encoded
– 12 special Symbols (K-Codes)
are used for framing and link
management (e.g. COM, STP,
SDP, etc)
– Scrambling 2^8 LFSR
PCIe Gen3 Encoding
– Use Scrambling for edge density
– Encoding scheme is 128/130
• no K-Codes any more for
framing
• Each 128/130 bits is a Block
– New tokens for framing and link
management
AMF 2010Page 36
Transmitting Across a Multi-lane Link
Sync Char
Symbol 0
Symbol 1
Symbol 2
Symbol 3
Symbol 4
Symbol 5
Symbol 6
~Symbol 15
Sync Char
Symbol 0
Symbol 1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
STP STP STP STP TLP HDR TLP HDR TLP HDR TLP HDR
TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR
TLP HDR TLP HDR TLP HDR TLP HDR Data Data Data Data
LCRC LCRC LCRC LCRC SDP SDP Payload Payload
Payload Payload LCRC LCRC LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDLLIDL LIDL LIDL LIDL
STP STP STP STP TLP HDR TLP HDR TLP HDR TLP HDR
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR TLP HDR
TLP HDR TLP HDR TLP HDR TLP HDR Data Data Data Data
TimeLane 0 Lane 1 Lane 2 Lane 3 Lane 4 Lane 5 Lane 6 Lane 7
AMF 2010Page 37
Analyzing Data on the Protocol Analyzer (Con’t)
Highlight block
structure clearly
STP STP STP STP
HDR HDR HDR HDR
HDR HDR HDR HDR
HDR HDR HDR HDR
00 00 00 00
LCRC LCRC LCRC LCRC
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
SYN[01] SYN[01] SYNC[01] SYN[01]
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
00 00 00 00
LCRC LCRC LCRC LCRC
00 00 00 00
00 00 00 00
SYN[10] SYN[10] SYNC[10] SYN[10]
AMF 2010Page 38
Finding Low Level Errors – Store Logical IDLEs
STP STP STP STP
HDR HDR HDR HDR
HDR HDR HDR HDR
HDR HDR HDR HDR
00 00 00 00
LCRC LCRC LCRC LCRC
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
SYN[01] SYN[01] SYNC[01] SYN[01]
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
SKP SKP SKP SKP
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
LIDL LIDL LIDL LIDL
00 00 00 00
LCRC LCRC LCRC LCRC
00 00 00 00
00 00 00 00
SYN[10] SYN[10] SYNC[10] SYN[10]
LIDL
AMF 2010Page 39
Debugging Speed Change Errors
Lane viewer tracking 2 different speeds:
Gen1 & Gen2 in the same view
AMF 2010Page 40
Framing Tokens
STP Token• Length of TLP embedded in the STP token (no
explicit END) in the TLP
• Length of total TLP including token (DW)
• STP Length field protected through a FCRC and a
Parity Bit
3 24 1 0567 3 24 1 0567
TLP LEN[10:4]
3 24 1 0567
FCRC
3 24 1 0567
00000000b
+0 +1 +2 +3
LEN[3:0] 1111b P
Logical Idle Token• All subsequent lanes in the
same symbol time should be
LIDL as well
3 24 1 0567
00000000b
+0
3 24 1 0567
11110000b
+0
3 24 1 0567
10101100b
+0
SDP Token
• Fixed pattern, always 0xF0AC
AMF 2010Page 41
Analyzing Data on Protocol Analyzer
AMF 2010Page 42
Protocol Layer Changes and Challenges
• Probing challenges due to the physical layer
• Protocol encoding & encapsulations changes
AMF 2010Page 43
Recovery Sub-State State Machine
Recovery.RcvrLock
Recovery.RcvrCfg
Recovery.Idle
Recovery.Speed
Exit to
DetectExit to L0
Exit to Hot
Reset
Exit to
Disable
Exit to
Loopback
Exit to
Configuration
Recovery.Equalization
AMF 2010Page 44
Recovery.Equalization Sub-State Behavior
Phase 0
Upstream
Component
TS1 : EC = 00,
preset EQ
TS1 : EC = 01,
preset EQ
Phase 1 TS1 : EC = 01,
preset EQTransition to Phase 1, if all
lanes receive at least 2
consecutive TS1 : (EC=01)
Phase 0
Downstream
Component
Transition to Phase 2, if all
lanes receive at least 2
consecutive TS1 : (EC=01)
TS1 : EC = 10,
Preset EQ
Phase 2
Phase 1
Phase 2
Transition to Phase 2, if all
lanes receive at least 2
consecutive TS1 : (EC=10)
Phase 3
Downstream component
requests a change in the
coefficients at the transmitter
of the Upstream componentUpstream component
requests a change in the
coefficients at the transmitter
of the Downstream
component
Transition to Phase 3, if all
lanes receive at least 2
consecutive TS1 : (EC=10)
Phase 3
TS1 : EC = 10, preset
EQ
Transition to Phase 3, if all
lanes receive at least 2
consecutive TS1 : (EC=11)Transition to
Recovery.RcvrLock, if lanes at
optimal settings
TS1 : EC = 11, EQ
from received TS1s
Transition to
Recovery.RcvrLock, if receive at
least 2 consecutive TS1:
(EC=00) Recovery.
RcvrLock
Recovery.
RcvrLock
TS1 : EC = 11, EQ
from received TS1s
TS1 : EC = 00, EQ
from received TS1s
AMF 2010Page 45
Exerciser as Root Complex & End Point
Exerciser Emulating EP Exerciser Emulating RC
Motherboard
Customer PCIe EP
Agilent
HardwareUp to X16 bi-
directional
7645
AMF 2010Page 46
LTSSM Test Case : Speed Change Gen1 to Gen3
AMF 2010Page 47
LTSSM Test Case : Results
AMF 2010Page 48
Protocol Layer Testing Key Considerations
1. Ensure that protocol tools for Gen3 have the right
physical layer characteristics to reliably capture data at
8GT/s
2. Software tools to help isolate common problems such as
speed change
3. Look for a known device to test your design against, and
exerciser
AMF 2010Page 49
AMF 2010Page 50
For further information
You will find more information on PCI Express 3.0 and Agilent solutions for
PCI Express at:
www.pci-sig.com
www.agilent.com/find/pciexpress
www.agilent.com/find/pcie3
www.agilent.com/find/si
PCI-SIG Website, Specification, S/W Tools,
Agilent Test Procedure
Agilent tools to help you succeed with your
PCI Express design such as the N5393A
Compliance application.
Agilent protocol test tools for PCI Express
3.0
Agilent tools to help you master signal
integrity challenges.