1

2

T-110.6120 Publish/Subscribe InternetworkingHelsinki University of Technology, Spring 2010

Lecture 5: Overview of the PSIRP architectureby Arto Karila

Slides based on the PSIRP project review slides presented in Brussels on March 3, 2009 and deliverable D2.3: Architecture Definition, Component Descriptions, and Requirements

3

Introduction

• Project objectives

• Consortium

• Working method

• Development process

4

Project Objectives

The key objectives of the PSIRP project are:1. Define and develop a new inter-networking architecture based on the

publish/subscribe (pub/sub) paradigm2. Develop a reference implementation of the core of the new architecture3. Analyse and measure the communications efficiency of new paradigm

by quantitative parameters4. Evaluate qualitative parameters of the new architecture with focus on

security and business incentives and models5. Develop migration and deployment strategies

Other aspects:• Follow the clean-slate design approach – take nothing for granted• Consider migration, e.g., via overlay solution• Emphasize security and socio-economics• Publish the results, including source code, as much and as widely as

possible • Engage with Future Internet community, e.g., cooperate with FIRE

(Onelab2) to test on large scale

5

Consortium

• The consortium was formed by people that shared a common vision and wanted to work together towards it

• The partners of PSIRP are:1. Helsinki University of Technology – Helsinki Institute for

Information Technology (TKK-HIIT, FI)

2. RWTH Aachen University (RWTH Aachen, DE)

3. British Telecommunications Plc (BT, GB)

4. Oy L M Ericsson Ab (LMF, FI)

5. Nokia Siemens Networks Finland Oy (NSNF, FI)

6. Institute for Parallel Processing of the Bulgarian Academy of Science (IPP-BAS, BG),

7. Athens University of Economics and Business (AUEB, GR),

8. Ericsson Magyarorszag Kommunikacios Rendszerek K.F.T. (HU)

6

Working Method

• Iterative working method– Define a pub/sub-based inter-networking architecture– Implement and validate the architecture– Revise the architecture and its implementation

• Work packages and tasks are allocated to teams that are formed spontaneously, across organizational boundaries– The purpose is to release creativity in a team consisting

of people of different companies and nationalities

The approach has proven its strength and it will be continued in the proposed Pursuit project

7

InitialPlanning

Planning

Requirements Analysis & Design

Implementation

Deployment

TestingEvaluation

Development Process

8

Technical Overview• Are the fundamentals still valid?• Observation: It’s all about information• Hypothesis: Increased information requires

information-centric network approaches• Approach: Clean-slate with late binding• Design methodology• Main design principles• Information centrism is key• Information concepts• Grouping information networks• Information structures

9

Are the Fundamentals Still Valid?

Fundamentals of the Internet• Collaboration

– Reflected in forwarding and routing

• Cooperation– Reflected in trust among

participants• Endpoint-centric services

– (mail, FTP, even web)– Reflected in E2E principle

IP, full end-to-end reachability

Reality in the Internet Today

• Phishing, spam, viruses

– There is no trust any more!

• Current economics favor senders

– Receivers are forced to carry the cost of unwanted traffic

• Information-centric services

– Do endpoints really matter?

– Endpoint-centric services move towards information retrieval through, e.g., CDNs

IP with middle boxes & significant decline in trust in the Internet

vs.

10

Observation: It's All About Information

Internet Today:• In 2006, the amount of digital information

created was 1.288 X 10^18 bits • 99% of Internet traffic is information dissemination & retrieval (Van Jacobson)

• HTTP proxying, CDNs, video streaming, …

• Akamai’s CDN accounts for 15% of traffic • Between 2001 and 2010, information will increase 1million times from 1 petabyte (10^15) to 1 zettabyte (10^21)• Social networking is information-centric • Most solutions exist in silos

• overlays over IP map information networks onto endpoint networks

Internet Today:• In 2006, the amount of digital information

created was 1.288 X 10^18 bits • 99% of Internet traffic is information dissemination & retrieval (Van Jacobson)

• HTTP proxying, CDNs, video streaming, …

• Akamai’s CDN accounts for 15% of traffic • Between 2001 and 2010, information will increase 1million times from 1 petabyte (10^15) to 1 zettabyte (10^21)• Social networking is information-centric • Most solutions exist in silos

• overlays over IP map information networks onto endpoint networks

Internet Tomorrow:• Proliferation of dissemination & retrieval services, e.g.,

• context-aware services & sensors• aggregated news delivery• augmented real life

• Personal information tenfold in the next ten years (IBM, 2008)• Increase of personalized video services

• e.g., YouTube, BBC iPlayer• Vision recognized by different initiatives & individuals

• Internet of Things, Van Jacobson, D. Reed

• Lack of interworking of silo solutions will slow innovation and development speed

Internet Tomorrow:• Proliferation of dissemination & retrieval services, e.g.,

• context-aware services & sensors• aggregated news delivery• augmented real life

• Personal information tenfold in the next ten years (IBM, 2008)• Increase of personalized video services

• e.g., YouTube, BBC iPlayer• Vision recognized by different initiatives & individuals

• Internet of Things, Van Jacobson, D. Reed

• Lack of interworking of silo solutions will slow innovation and development speed

11

Hypothesis: Increased Information Requires Information-centric Network Approaches

Application developers care about information concepts

– Creation of information topologies of various kinds

-> Endpoint-centric networking structures are inadequate

– Topological network changes too slow in timescale

– Topological network boundaries too restrictive

– Topological network boundaries often not aligned with information topologies

– Overlaying possible but restricted in (developer) scalability

-> If it is all about information, why not route on information?

12

Approach

Clean-slate design…• Question ALL fundamentals• Challenge our thinking• Take nothing for granted, including industry structures• Clear vision

…with late binding (to reality)• Consider migration and evolvability in separate work items

– How to get our design into real deployments, e.g., overlay vs. IP replacement?

• Even consider necessary evolution of industry (& regulatory) structures– How do industries need to evolve in certain scenarios?

13

Design Methodology

• Combine bottom-up and top-down– Implementation

matters but rationalization is necessary

• Adaptive design is crucial– Information-centrism

is expected to help for areas like metadata & policy

• Aligns well with cycle-based project approach– Creates micro-cycles

of question/remove

Choice Choice

Goals

Principles

Design Patterns & Considerations

Components

Choice

Derive

Map

Specify

Implement

Instance

Quest

ion

Constraints

Rem

ove

Add/Remove Add/Remove

VISION

Deployment

Deploy & evaluate

Observe

SoA

14

Main Design Principles

• Information is multi-hierarchically organised – Information semantics are constructed as

directed acyclic graphs (DAGs)

• Information scoping – Mechanisms are provided that allow for

limiting reachability of information to parties

• Scoped information neutrality – Within each information scope, data is only

delivered based on a given (rendezvous) identifier.

• The architecture is receiver-driven – No entity shall be delivered data unless it

has agreed to receive those beforehand.

Communication Model

InformationHierarchies

Informationreachability/scoping

15

Information Centrism is Key

• Information is everything and everything is information

– Bootstrap other concepts, e.g., identity, policy, …,

• Scopes build information networks

• Policy is metadata

– So is scope!

• Producers and consumers need no internetwork-level addressing!

Father FriendSpouse Colleague

Scope Family

Scope Company A

Data: Picture

Data: Mail

Governancepolicy

Scope FriendsGovernance

policy

Governancepolicy

16

Information Concepts

• Information– Smallest something

• Information collections– Sets of semantically similar information

• Information networks– Sets of information under some common

governance• Information producer

– Entity publishing information to a particular network

• Information consumer– Entity subscribing to information in a particular

network

17

Grouping Information Networks

18

RId2RId2

Information collectionsalg RId

Information Structures

RId1 RId2Information items

SId1Information networksSId2

SId2SId2

alg SId

Private Information networks

SId2

19

Architecture• High-level architecture• Architecture process• Architectural entities• Identifiers• Algorithmic identifiers• Architectural processes• Architecture overview• Components• Application considerations• Conclusions

20

High-Level ArchitectureRP : Rendezvous pointITF : Inter-domain topology formationTM : Topology managementFN : Forwarding node

ITFITF

Topology

RPRP

Rendezvous

RendezvousNetwork

Net

wor

k A

rchi

tect

ure

Service Model

Helper

Error Ctrl

…

Fragmentation

Caching

TMTM

TM TM

Forwarding

ForwardingNetwork Forwarding

Network

ForwardingNetwork

ForwardingNetwork

FN

pubpub

pubsub

Apps

Nod

e A

rchi

tect

ure

21

Architecture Process

• The aim of the architecture work is to produce coherent design for a publish/subscribe internetwork– Principles, framework, components, internetworking– Networks of information– Using pub/sub in all parts of the protocol suite

• The architecture work is iterative– Interaction with implementation and validation

workpackages• Encompasses both clean-slate and incremental

overlay-based solutions• In practice work is done in a number of

architecture teams with participants from different work packages

22

Architectural Entities

• Information items– Any data that can be labelled with an identifier– Items can have different identifiers depending on the level of

abstraction• Application identifier, rendezvous identifier, forwarding

identifier• Algorithmic identifiers

• Information networks (or scopes)– Information items grouped into a network of information under a

scope– Scopes can be interpreted at various levels of abstraction

• Information subscribers and publishers• Domains

– Administrative network areas that can be connected using inter-domain forwarding architecture

23

Identifiers

Publish / Subscribe

Metadata(source is implementation-dependent)

Data

Scope Identifiers(SId)

Includes...

Associated with...

Application Identifiers(AId)

Rendezvous Identifiers(RId)

Forwarding Identifiers(FId)

Network Transit Paths

Includes...

Resolved to...

Resolved to...

Define...

25

Architectural processes

• Rendezvous– The process of resolving higher level identifiers to lower

level identifiers within a given scope.– Three simple cases: link-local, intra-domain, inter-domain

• Topology management and formation– Management of data delivery topologies and forwarding

graphs• Forwarding

– Data delivery within a single administrative domain or across multiple domains.

– Temporal forwarding identifiers for each publisher and subscriber are derived via the rendezvous and topology management processes

– Various routing and forwarding protocols can be used, for example a new protocol replacing IP or IP-based overlays

26

Architectural processes

• Helper functions– Extensions to core functionality of the network

architecture, such as management and transport• Network attachment

– Discovery of network attachment points and network configuration

27

Subscriber Publisher

Forwardingedge nodes

Forwardingnode

Forwardingnode

Topology

ASRendezvous

Data Forwarding

Subscribe

Forwardingnode

Topology

ASRendezvous

Forwardingnode

Publish

Createdeliverypath

ConfigureForwardingpath

Architecture Overview

AS

Topology

28

Component Wheel

• The network architecture is based on a modular and extensible core called the PSIRP Component Wheel

• Components may be decoupled in space, time, and context– Layerless protocol suite

• Typical components include rendezvous, forwarding, helper functions, and caching.

• Applications may insert or request new components to the wheel at runtime.

• The components are attached to the local blackboard– Shared components, publications, state– Pub/sub is used to signal changes to blackboard state

29

Component Wheel

30

Compare w/ Haggle Architecture

[Sco2006]

J. Scott, P. Hui, J. Crowcroft, C. Diot, Haggle: A Networking Architecture Designed Around Mobile Users,

IFIP WONS 2006, Les Menuires, France, 2006

31

Component Wheel Interactions

Subscriber Blackboard Caching Routing and forwarding Low-level Link

Register and subscribe notifications

Register and subscribe notifications

Subscribe

Notify first component in plan

Component done, no cache hit

Notify second component in plan

Access memory object

Forward subscription

Publication

Insert publication as memory object

Notify componen in plant

Notify that has been added to cache

Notify subscriber

32

Service Model and API

We have considered four different classes of network services:1. A low-level page model that exposes network

forwarding and rendezvous/topology formation functions• Basic building block on top of raw forwarding API

– Pages and no error or congestion control• Listen(FiD), Forward(FiD, meta-data, data)• Mapping of information items to FiD• Pub/sub API towards higher levels

2. A mid-level memory object model• Memory pages are mapped to publications

3. A mid-level channel model4. Various high-level service models including shared

state and document models

33

APIs

Low-level page API

Memory Object API

Channel API

Higher-level APIs

34

Rendezvous

• Many faces of rendezvous– Link-local, inter-domain, information services, communal

services, …• The main requirements for this functionality are:

– Scalability to Internet-like networks and data space sizes– Efficiency of operation, measured in signalling overhead

and overall latency– Deployability

• The network is defined in terms of domains and their interconnections– Interconnections between domains include upstream,

transit, downstream– Rendezvous networks are units of deployment for the

PSIRP rendezvous functionality. The rendezvous networks are formed by rendezvous nodes (RNs), organized as a BGP-like inter-domain hierarchy

36

Rendezvous Networks

RNA

RNB

RNB10

RNB100

NSub

RNC

NPub

RP

RNC10

RNC100

Interconnection overlay

Rendezvous network

RNX Rendezvous node

End nodeNX

RP

Sub

Pub

Rendezvous point

Subscriber

Publisher

Rendezvous signaling

37

Intra- and Inter-Domain Operation

• Forwarding is configured by the rendezvous system with the help of the topology management function

• Key challenge is scalability• Intra-domain forwarding

– Initial work on Bloom-filter based forwarding mechanism indicates that they are useful for sizes up to metropolitan area networks

– FiDs identify partial distribution graphs– Minimal state in the routers

• Inter-domain forwarding– Solution must take performance and policy requirements

into account– Initial work on DHT-based overlay and understanding

implications for inter-domain routing– Inter-domain topology function

38

Security and Packet Level Authentication (PLA)

• Direct and indirect cryptographic association using identifiers and rendezvous– Public keys and one-way hash values– Algorithmic identifiers

• Packet Level Authentication (PLA) is one candidate protocol for securing PSIRP network functions– We assume that per packet public key cryptography operations are

feasible in Internet's scale because of new digital signature algorithms and advances in semiconductor technology

– PLA is a novel solution for protecting the network infrastructure against various attacks (e.g., DoS) by providing availability

– Each packet has a signature (ECC)– Good analogy for PLA is a paper currency: anyone can verify the

authenticity of the bill by using built-in security measures like watermark and hologram, there is no need to contact the bank that has issued the bill

– The network should be able to fulfill its basic goal: to deliver valid packets of valid users in reliable and timely manner in all situations

39

Prototype Implementation

PSIRP Apps

Legacy Apps

Sockets APIEmulator

Apps

PSIRP Library API

Upper layers(possibly Python)

Libraries

Kernel

PSIRP Kernel API

Lower layers (C/C++)

Ethernet Wi-Fi IP GMPLS

Packet Level Authentication (PLA)

40

Application Considerations

• Application programming interfaces– Inherently information centric– 1-to-1 message stream

• Similar to TCP/IP socket API– 1-to-N bidirectional connection

• Similar to UDP over IP multicast– 1-to-N document distribution

• Similar to CDN and P2P protocols

• Examples– WWW functionality– BitTorrent-like content distribution

41

Example: Scoping in CS-comms.

Using a scope to initiate client-server-like communications

42

Example: Invitation in CS-comms.

Using an invitation to initiate client-server-like communications

43

Example: Adding a Node to Tree

Simplified signaling for adding a node to a concast-like tree

44

Example: Interactions in the component wheel

Interactions in the component wheel

45

zFilter Forwarding Tables

46

Link ID Tag (LIT) Generation

47

LIT Router Model

48

Multicast-Assisted Mobility

49

Conclusions

We have outlined an information centric network architecture• Publish and subscribe are the basic primitives making

multicast the norm• Decoupling in space and time• Receiver driven (subscriber has control)• Rendezvous as the primitive to connect publishers and

subscribers across domains on multiple levels– Mapping to forwarding structures

• Scoping of data into manageable sets• Architecture work is iterative• Implementation and evaluation are on-going activities