sain networking

SAIN NetworkingSAIN = Synchronized Adaptive INfrastructure

Ray W SandersChairman, SAIN Networks, Inc.

overcoming unintended consequences

in voice and data networks

What this talk is aboutA simple paradigm that can overcome the unintended consequences of today’s stochastic data network.

The paradigm results in a simple underlayer that can

ensure a deterministic data network. 2

Sanders PredictionPackets will be forever, but the global Internet will morph into something that looks a little like a late 1970’s telephone network but with far more capability and without the

fatal flaws of carrying only connections that must last for at least a few secondsand support only voice

conversations3

4

Comparing SAIN with existing networks

Existing NetworksRoute one-connection-at-a time

Hop-by-hop routing for each connection

Network uses multiple control planes

Networks are largely stochastic

Wire speed latency can be 100 ns

Internet uses many overlay protocols

Data sent in bursts

Complicated Quality of Service required

Head-of-line blocking complications

Tough privacy and security problems

Bursty data can require overprovisioning

Many networking protocols must exist

SAIN NetworksRoute aggregations of connections

Route aggregations for a one-hop channel

Network uses a single control plane

Networks are deterministic

Latency inversely proportional to data rate

NICs make use of single purpose utility

Bursts forwarded or smoothed out

Guaranteed delivery—one metric: delay

Small packet wins by increasing data rate

Disjoint objects conceal cellet relevance

Aggregated streams require less BW

2 simple algorithms manage BW and routes

A thought experimentAssumptions:1.1,000 people want to see 1,000 two-hour movies starting at 8 p.m.2.Each movie contains 9 gigabytes of data3.The network can use up to 10 Gbps to deliver a collection of movies4.Suppose we use 10 Gbps for each movie

It takes 7.2 seconds to send one movieHow long does it take to send all 1,000 movies one after another?

1,000 × 7.2 seconds= 2 hoursHow long would the average customer need to wait to start seeing his

movie?1 hour

Now, suppose that we send each movie at 10 Mbps (1/1000th of 10 Gbps)How long does each of 1,000 customers wait to start watching his movie?

0 hoursThere is a compelling requirement to control bandwidth, (and hence delivery time) to meet each customer’s need

This result can obtain if a network is deterministic

5

Goals for a SAIN networkDefine and build elemental pieces

of a network architecture that:1. can support all existing voice and data network

traffic2. can support unknown future traffic types3. can grow from data centers,

to metropolitan networks, to a global interconnected network

4. is robust, efficient, and simple 5. is a circuit-based architecture that can endure

and scale for decades6

7

Constraining networks toreally improve their efficiency

A core principle of the SAIN architecture

Partition a network into small disjoint pairs of active objects such as

pairs of NICs and pairs of switches

Basic Aggregation / Disaggregation Switch Pairs

Generic Disaggregation Switch

Generic Aggregation Switch

Interconnecting Elements

What does this do?•Enhances a network’s privacy and security•Prevents one object in a network from changing the state of another without using a Control Vector to send messages from a source object to a destination object•Prevents any entity outside a network from changing the state of an object inside the network•Simplifies object addressing

8

Another core principle of a SAIN architecture

Nodes in a SAIN network are synchronized to a common clock


What does this do?•Enables very cheap high-performance switches that can scale well beyond current limits•Removes the need for complex Quality of Service facilities inside a network

9

A third core principle of the SAIN architecture

All user data protocolsare separated from

data transportand its control


Host,Terminal,Server, orNetwork

204

204


SAIN

Pr

otoc

olTr

ansl

ato

r

SAINUnderlayNetwork

Ingress | Egress NIC

SAIN

ProtocolTranslator

Ingr

ess

| Eg

ress

NIC

SAIN

Pro

toco

lTr

ansl

ator

What does this do?•Defines an underlay network whose only job is to transfer bits from a data source to a data sink•Enables Network Interface Controllers (NICs) to support devices with any protocol. •Demands that an Egress NIC’s protocols must match its paired Ingress NIC’s protocols•Lets a NIC match from only one other NIC to a large number of NICs in a network

10

A fourth core principle of the SAIN architecture

Build a lot of a network’s physical and logical connectivity a priori

to its use


What does this do?•Enables each port of a network to have a physical connection to every other port of the network with a matching NIC; the connections are set up when the network is built or modified•Enables every possible route to be computed when the network is built and need not be recomputed until new nodes are added to the network

11

A fifth core principle of the SAIN architecture

All connections are ‘virtual’ that consume network bandwidth only when there are data bits to transport


What does this do?•Enables each connection to be set up prior to use•Assures that no bandwidth is used until data is to be sent•Assures that an amount of bandwidth allocated to a connection is just enough to meet a customer’s needs

12

A sixth core principle of the SAIN architecture

All connections from a source node to a destination node are aggregated

into a single logical data flow


What does this do?•Significantly reduces the number of objects to be routed through a network•Packets do not get routed independently ; they are combined into aggregations sent from a source to a destination node through preset routes•No computing needed at each tandem node•A route is a virtual connection between two nodes; if it approaches congestion, another route can be quickly added

13

A seventh core principle of the SAIN architecture

The amount of available bandwidth and delay must be known for each possible route through a network

before a connection is made


What does this do?•Prevents discarding packets because of network congestion•Dynamically provides the most cost-effective route with bandwidth to meet each a connection’s need

How can this be accomplished?•Delay over a route is known when nodes are installed•Each node connected to a transport connection (trunk) sends the trunk’s bandwidth availability to each source node in the network periodically (e.g. 1,000 times per second)

What are data networking’sunintended

consequences?Some examples of unintended consequences

in today’s networks

1.Traffic congestion and discarded packets

2.Jitter (= delay variation); traffic shaping and policing

3.Overprovisioning and Quality of Service

4.Flow-based traffic and circuit emulation

5.Lack of privacy, security and survivability14

Packets and packet buffers are not going away in a SAIN network

For each end-to-end connection there is a packet buffer at its ingress node and one at its egress node

Each connection that occurs at a source-node/destination-node pair within a given period (an ‘epoch’ for a group of connections)

originates within a pair of switches

15

Network Behavior Constraint 1Eliminate Traffic Congestion

Basic Source Aggregation / Destination Disaggregation Switch Pairs




Interconnecting Elements include source/destination node switches

in three aggregation tiers above the lowest tier

The lowest tier aggregates customer data; the higher tiers forward aggregations

Each higher tier aggregates the next lower tier’s data

16

Network Behavior Constraint 2

Eliminate jitter, traffic shaping and policing

Jitter (also known as delay variation) is the aperiodic arrival of each packet. Aperiodic arrivals of packets in

data flows can cause service disruptions

Changing bandwidth of a connection can assure that either the start time of a received packet or the

time required to receive an entire packet provides uninterrupted service

SAIN network synchronization provides ‘traffic shaping’ and ‘policing’ without additional complexity

17


Reduce overprovisioining

Aggregating connections into channels can benefit from the Law of Large Numbers

The law can result in the bandwidth of a large aggregation changing slowly

compared to faster bandwidth changes of the lowest tier

Node synchronization can result in a network not needing Quality of Service

as currently defined

A desirable metric is end-to-end delay of entire packets—not wire speed starting time of sending a single packet

18


Flow-based traffic without Circuit Emulation

Nodal clocks can provide physical circuits in a simple mannercompared to the current complexity

of circuit emulation

The physical circuits operate at all levels of aggregation and can be virtual or real

The necessity of providing circuits for flow-based traffic is a major reason to implement the SAIN architecture

In addition to basic algorithms, a third ‘floating frame’ algorithm exists for

plesiochronous operation where span lengths of trunks vary (e.g., for moving nodes and

environment variations)

19


Provide better privacy, security and survivabilityOvercome current core network privacy and security weaknesses

A SAIN network can assure that all network objects used to forward packet data through the network are disjoint.

Network data forwarding control can be massively distributed with centralized monitoring and fault management

A network object cannot change the state of another object except by using a certified Control Vector connected from

a source node to a destination node

A destination node can authenticate certification of a connected Control Vector.

Certification can use round-trip delay of destination and source nodes

Bandwidth management algorithm results in ever-changing aggregation frames

‘Floating frames’ enhance security

20

Architecture scales beyond current limits

More Network Behavior

Instead of forwarding entire packets a SAIN network forward only one or a few bits of a packet at a time

This results in using very simple switches that forward large aggregations without requiring

expensive large routers

Not only are costs reduced;energy needs are reduced as well

There is no need for traffic shaping or policing;there is no need for circuit emulation;

there are no out-of-order packets; and the packet loss rate is zero

Synchronized network nodes and implicit addressing achieves this goal

Node synchronization can result in a single metric that defines required

delays for application types

The single metric defines end-to-end delay of entire packets—not just the wire speed starting time

of sending a single packet

21

A single metric defining application needs


22

Results from simulations of a model network


A Metropolitan Area Network Example with 20 T-Nodes & 80 Simplex Trunks500 E-Nodes each able to support >4,000 ports each with multiple IP addresses

20

1519

14

10

7

4

1

18

13

9

63

16

12

8

5

2

11

015

016

247.

0

002

001

248.8

004

003

367.

9

005 006

217.5

009

010

278.

0

007 008165.8

233.8

013 014

012 011

200.4

078 077185.5076

0

75

279.7

021

0

22

204.1

020

019

317.9

291.4

018 017

023 024

198.5

281.

1

027

028

026

025

273.6

029 030217.4

31 032

265.9226.

203

4

033

041

042

226.

3

035 036

269.2

037 038

330.7

039

040

415.2

044

0

4317

2.1

330.1

045 046

050 049163.3

056

0

55

203.2

052

051

214.

5

053 054

238.5

047

048

229.2

057 058

294.7

060 059

215.5

067

0

6829

5.8

063 064

161.8

061 062246.2065

066

158.2

069

07

035

7.9

072 071

188.8

074 073

272.1

80

79

423.

5

17

19

The green circles are transit nodes (T-Nodes)The red rectangles are entry/exit nodes [E-Nodes]Each [E-Node] (T-Node) contains source switches connecting to paired destination switches in all other [E-Nodes] (T-Nodes) in a network

Sanders SuggestionWe should not let ourselves make another

management mistake that the future of networking will be based entirely on

using packet switches for routing

Our focus should morph into efforts that enhance IP* addressing and DNS*

in a circuit-based world with advanced NIC applications

23* Internet Protocol addressing and

* Domain Name System

• Transport of bits is independent of data type

• Packets appear only at ingress and egress ports with connected NICs• Packet or circuit data appears at an ingress NIC and is transferred

to an egress NIC• An ingress/egress pair of NICs can support any matched data type• NIC pairs can support secure topologies and methods• Packets are transferred bit-by-bit at a deterministic data rate• An Egress NIC delivers the protocol entering its paired Ingress NIC

26


204

204


SAIN

Pr

otoc

olTr

ansl

ato

r

SAINUnderlayNetwork

Ingress | Egress NIC

SAIN Protocol

Translator

Ingr

ess

| Eg

ress

NIC

SAIN

Pro

toco

lTr

ansl

ator

How to support goals 1 & 2 (Support existing and future traffic types)

How SAIN works #1What a packet flow can look like:

27

Packet HeaderH

Packet DataD

H D

H D H D

H D

H DH DInput A

Input BA1 A2 A3 A4

B1 B2

Output H D H D H D H D H DH DA1 A2 A3 A4B1 B2

This method of multiplexing uses ‘explicit addressing’

What a SAIN flow can look like:

The size of each cellet is fixed for a given link in which a frame occurs

The duration of an Epoch can depend on the desired end-to-end network delay of all embedded packets

This method of multiplexing uses ‘implicit addressing’ where the position of each cellet defines its connection or channel identity

How SAIN works #2

28

A SAIN network containssimple network switches with a

very different approach that uses very simple parts

The ‘Interconnecting Elements’ are primarily made up of Aggregation Switch / Disaggregation Switch pairs

that exist in three levels of aggregation

Each tier contains Aggregation / Disaggregation Switch Pairs

The three aggregation levels pass data use three network tiers plus an exchange tier to other networks and a virtual

distribution sub-tier shown in the next slide

Basic Aggregation / Disaggregation Switch Pairs




29

How SAIN works #3Connections exist in an Entry/Exit E-Node tier that includes a virtual VE-Node subnetwork uses for traffic distribution

Each E-Node connects large aggregations of connections within large channels to and from a parent Transit T-Node tier

In addition to its T-Node tierrouting functionality, a T-Node can connect to an eXchangeX-Node that can have a channel to other X-Node domains including those that make up a global domain

Each T-Node routes the aggregations of E-Node traffic for delivery from a SourceT-Node to a Destination T-Node

eXchangeX-Nodes

Transfer T-Nodes

Entry/Exit E-Nodes

Virtual Entry/Exit NodesVE-Nodes

Each E-Node connects to a parent T-NodeEach T-Node has full period connections to every other T-Node

Each Source T-Node can set up a loop-less route through T-Nodes to every other T-Node

Each route can be computed at network instantiationThe computation begins with a table of

single hops among the T-NodesA second hop for each entry can be added for each second hop

that does not include the first hopRepeat this process recursively for a two-hop table to build a

three-hop table and continue for tables with more hopsThe process results in finding all routes that do not contain loops

A 10-hop table has over 500,000 entries for all source to destination routes in a 20 T-Node model network

The average number of routes for each of the 380 paired connections is about 1300

30

Routing in the model network


31

Today’s networks are based on early 1970’s needs: using minicomputers to

send messages and transfer files

Queuing theory provided solutions for an asynchronous stochastic world

Today’s needs are circuit-based tosatisfy a burgeoning market

for flow-based traffic

What is needed now is a network with synchronized nodes that support

dynamic data rate connections

Today’s traffic is mostlyflow-based, not bursty

32

Another experimentAssumptions:1.A financial trading firm wants to minimize its network delay2.The smallest Ethernet frame is 84 bytes including a 46-byte payload3.A SAIN network frame can have 5 bytes plus the 46-byte payload4.In either case, a 1 Gb/s channel is carrying the dataA SAIN 408-bit (51-byte) packet could be guaranteed delivery in one microsecond or less This compares to 672-bit (84-byte) Ethernet needing nearly one microsecond if there is no other traffic using the channel. Its delay is not guaranteed.

There is a compelling requirement to control bandwidth, (and hence delivery time) to meet a customer’s need

3333

A brief look at a basic principlethat really matters

10 nsec

100 nsec

1.0 μsec

10 μsec

100 μsec

1.0 msec

10 msec

100 msec

1.0 nsec1.0 kb/s 10 kb/s 100 kb/s 1.0 Mb/s 10 Mb/s 100 Mb/s 1.0 Gb/s 10 Gb/s 100 Gb/s

Delay vs. Data RateAn 8 x 8 orders of magnitude look at a key fundamental of

data networking

34

2 m

sec

optical

fiber

radiu

s

~400

km

Area of the Square ≈ 320,000 km²

Area of the Circle≈ 502,654 km²

~56

5.7

km

The earth’s land mass area totals ~148,940,000 sq km.The area of each square within a 2 millisecond

radius circle is ~320,000 sq km.The number of supermetro networks

needed to cover the land mass: 466

Can we cover the earthwith a SAIN network?

In the real world, sizes will likely be based on number of users and/or number of ports

and market to determine a diameter

sain networking

Documents

deterministic data network

network architecture

data networkswhat

data ratenics

data centers

s telephone network

gigabytes of datathe

data ratedisjoint objects