distributed virtual transaction directory server

DVTDS

Christian Hollstein, TeraCortex

www.teracortex.com

Presentation of DVTDS

Distributed Virtual Transaction Directory Server

By TeraCortex

● Background

● Architecture

● Virtualization

● Performance

Background: LDAP in Mobile Networks

MediaServerIMSAS

CSCF

IMSdomain

Provisioning System

LDAPTransactions

LDAP

LDAP

SGSN

GGSN

MSC

3GnetworkHLR

MME

HSS

4Gnetwork

3Com CoreBuilder 5000TM Switching Hub

mgt fb fb fb fb fb fb fb fbtpl6tpl6 5302m

SDMDirectory

LDAP

LDAP based Subscriber Data Management

● 3GPP standard rules LDAP as central repository

● Several hundred mobile operators / deployments worldwide

● Major vendors: Ericsson, Huawei, NSN, ZTE, Alcatel

● NSN alone serves 3.2 billion subscriber records

● Several dozen entries per subscriber record

● Probably largest directories worldwide

Consequences for Directory Products

● Millions of subscriber records → billions of entries

● Data federation / distribution

● High availability → geo -redundant deployment / replication

● Consistent provisioning → transaction safeness

● Update signaling to applications → triggers

● Multi application environments → data model virtualization

● High volume traffic → near real time behavior

DVTDS

New Solution Coming Up:

DVTDS Distributed Architecture

Client… > 1000

Client Client Client

DVTDS (chained)

LDAPChaining

...

LDAP

LDAPChaining

• LDAP protocol for chaining• Multi level hierarchy• Leaves may be any LDAP server• Sessions span over several servers• Servers may be replicated• Distributed transactions

Possiblesessionpath

DVTDS 1000 million keyson 64 GB machine (mirrored)

...

(chained, mirrord)

...

Data Replication

(Mirror 0)

LDAP Mirror

• Symmetrical Multi Master Replication• No single point of failure• Logical DSA concept• Compatible with LDAP chaining• Priority based conflict resolving, real time• LDAP protocol• Up to eight servers per DSA, fully meshed• Transaction safe

Logical DSA

Client

Sessionpath

LDAPconnection toany of themirrors

(Mirror 1)

(Mirror 2) (Mirror 3)

Replication and Conflict Resolving

DVTDSSite APrio 2

UserPrio 7

LDAPDeletePrio 0

LDAPMirror

• Conflicts recognized and handled in real time• Based on request, user and server priority• Keeps to ACID paradigm• Data consistent across sites under attack• Winner gets “Success”. Looser gets “Busy”

Sessionpath

DVTDSSite BPrio 5

LDAPModifyPrio 1

Sessionpath

UserPrio 4

ObjectResolverObject

ResolverObject

ResolverObject

Resolver

System Integrationand External Interfaces

...Client Port

AdminPort

LDAP

...Capture Port

Log FileTrigger...

...DataFederation

...DataReplication

...

Restore / DataMigration

...

Reports...

Backup / DataMigration

...

SOAP/HTTP

CSV

LDAP

LDAP LDAP

CSV

LDIF

CSV

LDIF

LDIF

CSVLDIF

LDAP

BinaryASN.1

OAMSystem

Applications /Provisioning

Internal Architecture

...Client Ports Capture Ports

Session ...

Protocol Stack

Object Resolver

Execution Unit

Protocol Stack

Object Resolver

Execution Unit

Protocol Stack

Object Resolver

Execution Unit...

Interfaces:TriggerBackupRestoreMigrationReportsAdminLog filesChainingReplication

ConfigurationSchemaBackup/RestoreTraffic controlTuningDNSLicensesLogging/Audits

Interlocking sub system

DirectoryInformation Tree

Central Data Area

DVTDS

Session, queuecontrol

Hard disk sub system

Architectural Features

● Free configurable client ports

● Each client port serves a number of sessions

● Each session lives inside its own worker thread

● Object level locking system

● Direct data allocation on memory mapped hard disk volumes

● Volumes maybe cooked or raw file space

LDAP Data Model Virtualization

HLRMMS

HSS

PCRFFixedNet

MNP

AAA IMS

M2M

Social Networks

Data access viaapplication views

Physical dataaccess (No views)

ViewLayer

Provisioning System

Application Data

CoreData

Supported LDAP View Mechanisms

● Transparent aliases

● Rule based bidirectional DN conversion

● Virtual objects

● Virtual and real attributes can be mixed in any object

● Soon: Rule based bidirectional attribute/value conversion

● Integrated in the DVTDS kernel → little overhead

● Online configurable → no service interruption

ou=MOBILE ou=FiXEDou=EMAIL

Data Aggregation by Virtualization:Physical Telco Model

UID=777888000000001

oc: inetOrgPerson

oc: imsiUidAlias

dc=IMSI

oc: dcObject

o=<BusinessUnit>

oc: organization

ou=subscriberData

oc: organizationalUnitt IMSI=777888000000001

Access Path

ou=IDENTITY

oc: imsiAlias

dc=IMSI

oc: dcObject

IMSI=262011100000001

oc: imsiUidAlias

dc=IMSI

oc: dcObject

IMSI=777888000000001

oc: msisdnAlias

dc=MSISDN

oc: dcObject

MSISDN=4916096220958

oc: imsiUidAlias

dc=IMSI

oc: dcObject

IMSI=777888000000001

oc: mailAlias

dc=EMAIL

oc: dcObject

mail=me@teracortex.com

dc=IMSI

oc: dcObject

dc=Enterprise

oc: dcObject

dc=IMSI

oc: dcObject

dc=configurableViews

oc: dcObject

oc: imsiUidAlias

dc=IMSI

oc: dcObject

IMSI=777888000000001

oc: accountlAlias

dc=ACCOUNT

oc: dcObject

account=1234abcd

dc=FIXED

oc: fixedNetDataparam4: real valueparam5: real value...Fixed Net: reference

dc=IDENTITY

oc: identityDataparam6: real valueparam7: real value...Identy: reference

dc=EMAIL

oc: eMailDataparam2: real valueparam3: real value...Email: referencedc=MOBILE

oc: mobileDataparam0: real valueparam1: real value...Mobile: reference

Fixednet data

SubscriberIdentities

Email DataMobile

Data

...

View Mechanism Properties

● Each subscriber has individual data below uid=...

● Accessed via transparent aliases

● Application view data outside of subscriber data

● Found by two stage resolving algorithm

● Different applications can share physical data

Example: Server – Side DN Conversion

Server Side Conversion Rule:

clientDn: *,impi=(sip):([0-9]+)@(ims.telekom.de),dc=IMPI

serverDn: imsi=#3(2),dc=IMSI

DN as sent by the client:

ou=mobile,impi=sip:262000000000000@ims.telekom.de,dc=IMPI

DN as used by the server:

ou=mobile,imsi=262000000000000,dc=IMSI

Throughput in absolute numbers

200000

Entryload

LDAPSearch

LDAPModify

Op

erat

ions

/ s

DVTDSIntel I7 4960X6 Cores @4.6 GHz32 GB RAM7 x SATA 7200 RPM28 Million entries

Oracle OIDSparc T5-232 cores @3.6 GHz512 GB RAMFlash disk array50 million entries

LDAPCompare

LDAPAdd

100000

300000

400000

500000

600000

700000

800000

900000

1000000

Throughput per GHz CPU speed

6000

Entryload

LDAPSearch

LDAPModify

Op

erat

ions

/ s

DVTDSIntel I7 4960X6 Cores @4.6 GHz= 27.6 GHz

Oracle OIDSparc T5-232 cores @3.6 GHz= 115.2 GHz

LDAPCompare

LDAPAdd

3000

9000

12000

15000

18000

21000

24000

27000

Throughput Scaling

Notes on 3D Server Throughput Diagram

● 2 Variables: queue length and number of clients

● Throughput increases with bigger queue length

● Throughput scales by number of cores and clients

● Saturation on 6 core machine at 6 clients

● Degradation when operated beyond saturation

● Linear scaling if not bottle - necked by memory bandwidth

Scaling the Data

• 540 million entries in less than 2 hours• Naming attribute was indexed• Indexing time included, no setup time• Multi threaded object loader• LDAP protocol / BER object format• 30 GB RAM, 366 GB data base size

540 Million entries

inetOrgPerson22 AttributesLDIF size: 532 bytes

114 Minutes load time

Linear sc

aling

Roadmap 2014

● Automatic replica reconciliation after mirror network faults

● Free configurable indices

● User level documentation

● Free demo version download

Thank you for your attention!

www.teracortex.com