CMPE 511CMPE 511

ON CHIP NETWORKS: A Scalable, ON CHIP NETWORKS: A Scalable, Communication-Centric Embedded Communication-Centric Embedded

System Design ParadigmSystem Design Paradigm

Giray KömürcüGiray Kömürcü

OUTLINEOUTLINE

System On Chip (SoC)System On Chip (SoC)

Network On Chip (NoC)Network On Chip (NoC)

Characteristics of NoC’sCharacteristics of NoC’s

Customization Potential for On-Chip Customization Potential for On-Chip NetworksNetworks

State of the Art Research in On Chip State of the Art Research in On Chip NetworksNetworks

Future Research DirectionsFuture Research Directions

System On ChipSystem On Chip

More and more transistors availableMore and more transistors available

All operations integrated on a single dieAll operations integrated on a single die

Higher level design methodologiesHigher level design methodologies

System On ChipSystem On Chip

New architectures:New architectures: Thousands of processorsThousands of processors Sophisticated communication architectureSophisticated communication architecture

Buses are not enoughBuses are not enough

Large bus lengths -> clock skew problemLarge bus lengths -> clock skew problem

Scalable communication architecture is Scalable communication architecture is neededneeded

Bus-Based SOCBus-Based SOC

Network On Chip (NoC)Network On Chip (NoC)

Success & Scalability of Internet inspired Success & Scalability of Internet inspired researchers for on chip communicationresearchers for on chip communicationSwitch based networks & packet based Switch based networks & packet based communicationcommunicationScalabilityScalabilityStandardizationStandardizationReuseReuseImportant for lowering design effort, time Important for lowering design effort, time to marketto market

NoC FeaturesNoC Features

Packets transmitted instead of wordsPackets transmitted instead of words

Dedicated address lines are not necessaryDedicated address lines are not necessary

Parallel transactionsParallel transactions

Clock skew and cross talk is no concern Clock skew and cross talk is no concern due to routersdue to routers

Structured wiring, easy routingStructured wiring, easy routing

Network On ChipNetwork On Chip

Characteristics of NoC’sCharacteristics of NoC’s

Wiring resources:Wiring resources: Abundant resources for wiringAbundant resources for wiring Thousands of wires at each metal layer Thousands of wires at each metal layer In non network based systems:In non network based systems:

Hard to Predict – Hard to Avoid Cross-Coupling Hard to Predict – Hard to Avoid Cross-Coupling effectseffects

Repeater insertionRepeater insertion

Conservative, Slow designConservative, Slow design

More regular distribution of power distributionMore regular distribution of power distribution

Characteristics of NoC’sCharacteristics of NoC’s

Wiring resources:Wiring resources: In on chip networks:In on chip networks:

Same length wires Same length wires

Predictable delaysPredictable delays

Predictable cross talk patternPredictable cross talk pattern

Non-conservative line width sizing and layoutNon-conservative line width sizing and layout

Higher clock ratesHigher clock rates

Characteristics of NoC’sCharacteristics of NoC’s

Traffic Patterns:Traffic Patterns: Poisson distribution in large scale networksPoisson distribution in large scale networks More predictable in embedded systemsMore predictable in embedded systems Pre-Scheduling increases the speed of Pre-Scheduling increases the speed of

transmissin and actual band-widthtransmissin and actual band-width

Characteristics of NoC’sCharacteristics of NoC’s

Heteorgeneity:Heteorgeneity: Several different algorithms on SoCsSeveral different algorithms on SoCs Heterogeneous communication loadsHeterogeneous communication loads Network architecture may be specialized to Network architecture may be specialized to

match the traffic reduces the size and power match the traffic reduces the size and power consumptionconsumption

Characteristics of NoC’sCharacteristics of NoC’s

Power ConsumptionPower Consumption One of the most important issues in embedded One of the most important issues in embedded

systemssystems NoC’s may worsen the problemNoC’s may worsen the problem Less power for each line but more wiresLess power for each line but more wires More communication due to many coresMore communication due to many cores Power consumption of the interconnect is not Power consumption of the interconnect is not

negligiblenegligible Removal of the address lines is a plusRemoval of the address lines is a plus Data is not broadcasted as in bus systemsData is not broadcasted as in bus systems

Characteristics of NoC’sCharacteristics of NoC’s

TestabilityTestability Reliability is no longer limited to critical Reliability is no longer limited to critical

applicationsapplications Three components are needed:Three components are needed:

Source for generating test stimuliSource for generating test stimuli

Test access mechanisim to move the test dataTest access mechanisim to move the test data

Test scheduling strategyTest scheduling strategy Design for test strategy to minimize the Design for test strategy to minimize the

volume(cost) and increase the coveragevolume(cost) and increase the coverage

Characteristics of NoC’sCharacteristics of NoC’s

TestabilityTestability NoC can be considered as a core NoC can be considered as a core Is composed of identical components and Is composed of identical components and

interconnects each component on the systeminterconnects each component on the system Hierarchical structure of NoC allows test re-useHierarchical structure of NoC allows test re-use Test data can be broadcast to all identical elementsTest data can be broadcast to all identical elements High speed interconnect testing will dominate since High speed interconnect testing will dominate since

many lines in the systemmany lines in the system Presence of communication protocol complicates Presence of communication protocol complicates

testingtesting Timing tests are important since clock boundaries of Timing tests are important since clock boundaries of

cores are in the network interfacecores are in the network interface

Characteristics of NoC’sCharacteristics of NoC’s

LimitationsLimitations Memory access between IP cores and memory cores Memory access between IP cores and memory cores

is a performance bottleneckis a performance bottleneck No standardizationNo standardization Customization may boost the performanceCustomization may boost the performance Hard to achieve large scale reusability of the Hard to achieve large scale reusability of the

communication backbone communication backbone Hard to create design independent communication IP Hard to create design independent communication IP

CoresCores Adoption of Network on Chips will be slowAdoption of Network on Chips will be slow

Designers are not expert on the network conceptDesigners are not expert on the network conceptCAD tools should be developedCAD tools should be developed

Customization Potential for On-Customization Potential for On-Chip NetworksChip Networks

Large scale networks have to be strictly Large scale networks have to be strictly standardized to serve each clientstandardized to serve each client

Minor issue in on chip networksMinor issue in on chip networks

IP core can be freely designed to fit any NoCIP core can be freely designed to fit any NoC

If standardized, IP cores from different vendors If standardized, IP cores from different vendors can be usedcan be used

Customization of networks may lead best Customization of networks may lead best performance, area and power consumptionperformance, area and power consumption

Architectural CustomizationArchitectural Customization

Network topologies can be characterized by:Network topologies can be characterized by: Number of links connecting a node to othersNumber of links connecting a node to others Maximum distnance between any two nodesMaximum distnance between any two nodes RegularityRegularity SymmetrySymmetry

Heterogenity of computation should be reflected Heterogenity of computation should be reflected in the NoC to prevent over designin the NoC to prevent over design

Buffer SizeBuffer Size

Number of Virtual channelsNumber of Virtual channels

Architectural CustomizationArchitectural Customization

Protocol CustomizationProtocol Customization

Communication Protocols: Rules and Communication Protocols: Rules and Methods required for transfer between IP’sMethods required for transfer between IP’sHandshaking via request & acknowledgeHandshaking via request & acknowledgeMore complex protocols in NoC’sMore complex protocols in NoC’sApplication Specific customization of Application Specific customization of protocols can provide huge power and protocols can provide huge power and performance benefitsperformance benefitsPacket size and Routing are two Packet size and Routing are two characteristics of protocol customizationcharacteristics of protocol customization

Protocol CustomizationProtocol Customization

Routing:Routing: Path origin to the destinationPath origin to the destination ConnectivityConnectivity Adaptivity Adaptivity Deadlock and livelock freedomDeadlock and livelock freedom Fault toleranceFault tolerance Is broadcasting requiredIs broadcasting required

Protocol CustomizationProtocol Customization

Packet size:Packet size: Finding the optimum is crucial for optimum Finding the optimum is crucial for optimum

use of network resourcesuse of network resources Depends on the characteristics of the Depends on the characteristics of the

application (data or control)application (data or control) Too small packets have overhead to reuniteToo small packets have overhead to reunite Too big packets may block traffic for long timeToo big packets may block traffic for long time Should be chosen according to buffer sizeShould be chosen according to buffer size

Protocol CustomizationProtocol Customization

Customization for Quality of Customization for Quality of ServiceService

Performane of the combination must be validatedPerformane of the combination must be validatedBehaviour of NoC and IP Cores should be abstracted to Behaviour of NoC and IP Cores should be abstracted to see service requirementssee service requirementsRetransmission or error correctionRetransmission or error correctionError correction quaranties minimum BWError correction quaranties minimum BWIf Data is not accepted by the IP data is lost if not If Data is not accepted by the IP data is lost if not enough buffersenough buffersA clear, service level specification of the NoC services is A clear, service level specification of the NoC services is necessarynecessaryFlow control is essentialFlow control is essentialNoCs must implement a resource management strategy NoCs must implement a resource management strategy to allocate BW and notifies the IPto allocate BW and notifies the IP

Customization in HW/SW Trade OffCustomization in HW/SW Trade Off

Router-Switch may be a part of local Router-Switch may be a part of local processorprocessor

Routing and arbitration algorithms may be Routing and arbitration algorithms may be assumed by the local processorassumed by the local processor

Virtual channels may be managed by the Virtual channels may be managed by the local processorlocal processor

Local Memory may be used instead of Local Memory may be used instead of buffersbuffers

State of the Art Research in On State of the Art Research in On Chip NetworksChip Networks

Architectural Issues:Architectural Issues: Adopting buffers efficiently to trade off Adopting buffers efficiently to trade off

performance and costperformance and cost Packetization of dataPacketization of data Designing router architecture to trade of cost Designing router architecture to trade of cost

and efficiencyand efficiency Complex architectures using NoC’sComplex architectures using NoC’s

Power Energy IssuesPower Energy Issues

Estimating a routers power consumptionEstimating a routers power consumption Sthocastic traffic models for transition activity, Sthocastic traffic models for transition activity,

packet arrival and departure eventspacket arrival and departure events Simulation based frameworkSimulation based framework Bit level energy estimation of routersBit level energy estimation of routers Interconnect models to compute power at Interconnect models to compute power at

system levelsystem level

Simulation IssuesSimulation Issues

Crucial to estimate performance before Crucial to estimate performance before implementationimplementation

Protocol and topology simulationsProtocol and topology simulations

Accurate vs fast simulationAccurate vs fast simulation

Future Research DirectionsFuture Research Directions

Few concrete implementations for complex Few concrete implementations for complex NoCsNoCsStandardizationStandardizationCycle accurate simulation is not appropriate Cycle accurate simulation is not appropriate since more than one proccessing unitssince more than one proccessing unitsCycle approximate simulations may developCycle approximate simulations may developCommunication centric simulation is neededCommunication centric simulation is neededHierarchical architectures to build efficient NoCs Hierarchical architectures to build efficient NoCs with hundreds of non uniform coreswith hundreds of non uniform coresCross talk problemCross talk problem

CONCLUSIONCONCLUSION

Communication architectures on chip become Communication architectures on chip become bottleneck with more and more transistorsbottleneck with more and more transistorsNetwork On Chip’s are a good solution for SoCs Network On Chip’s are a good solution for SoCs in the billion transistor erain the billion transistor era Packets transmitted instead of wordsPackets transmitted instead of words Dedicated address lines are not necessaryDedicated address lines are not necessary Parallel transactionsParallel transactions Clock skew and cross talk is no concern due to Clock skew and cross talk is no concern due to

routersrouters Structured wiring, easy routingStructured wiring, easy routing

Customization of NoC’s gives the oppotunity to Customization of NoC’s gives the oppotunity to achieve high performance in SoC’sachieve high performance in SoC’s

THANK YOUTHANK YOU