spinnaker networks, inc

of 30 NFS Industry ConferenceOctober 22-23, 2002

INDUSTRY

CONFERENCE

Spinnaker Networks, Inc.

www.spinnakernet.com

301 Alpha DrivePittsburgh, PA 15238(412) 968-SPIN

INDUSTRY

CONFERENCE

• ”Everything you know is wrong”– … at least eventually

– space requirements change

– “class of service” changes

– desired location changes

Storage Admin’s Problem

INDUSTRY

CONFERENCE

• System scaling– add resources easily

– without client-visible changes

• Online reconfiguration– no file name or mount changes

– no disruption to concurrent accesses

• System performance

Solution

INDUSTRY

CONFERENCE

Spinnaker Design

• Cluster servers for scaling– using IP (Gigabit Ethernet) for cluster links

– separate physical from virtual resources– directory trees from their disk allocation

– IP addresses from their network cards

– we can add resources without changing client’s view of system

INDUSTRY

CONFERENCE

Spinnaker Design

• Within each server– storage pools

– aggregate all storage with single service class– e.g. all RAID 1, RAID 5, extra fast storage

– think “virtual partition” or “logical volume”

INDUSTRY

CONFERENCE

nnaker

Archit

• Create virtual file systems (VFSes)– a VFS is a tree with a root dir and subdirs– many VFSes can share a storage pool

– VFS allocation changes dynamically with usage– without administrative intervention– can manage limits via quotas

– Similar in concept to– AFS volume / DFS fileset

INDUSTRY

CONFERENCE

nnaker

Archit

adam ant

BachBobs

net disk

DeptsUsers

A B Eng

Storage Pool

INDUSTRY

CONFERENCE

nnaker

Archit

• Create global “export” name space– choose a root VFS

– mount other VFS, forming a tree– by creating mount point files within VFSes

– export tree spans multiple servers in cluster– VFSes can be located anywhere in the cluster

– export tree can be accessed from any server

– different parts of tree can have different CoS

INDUSTRY

CONFERENCE

adam ant

net disk

Depts Users

A B Eng

Depts Users Users

adam ant

net disk

Storage Pool

BachBobs

INDUSTRY

CONFERENCE

Clustered Operation

• Each client connects to any server– requests are “switched” over cluster net

– from incoming server

– to server with desired data

– based on– desired data

– proximity to data (for mirrored data)

INDUSTRY

CONFERENCE

Cluster Organization

SpinServer cluster

Windows desktop

UNIX workstation

Windows desktop

SpinServer 1

SpinServer 2

SpinServer 3

Q1 eng P2

INDUSTRY

CONFERENCE

Client Access

Gigabit Ethernet

Fibre Channel

Client Access

Gigabit Ethernet

Fibre Channel

Disk Process• Caching• Locking

Gigabit EthernetSwitch

Network Process• TCP termination• VLDB lookup• NFS server over SpinFS

SpinFS Protocol

Network Process• TCP termination• VLDB lookup• NFS server over SpinFS

Disk Process• Caching• Locking

Server/Network Implementation

INDUSTRY

CONFERENCE

• At enterprise scale, security is critical– don’t have departmental implicit “trust”

• Kerberos V5 support– For NFS clients

– Groups from NIS

– For CIFS using Active Directory

Security

INDUSTRY

CONFERENCE

• A virtual server consists of– a global export name space (VFSes)– a set of IP addresses that can access it

• Benefits– additional security fire wall

– a user guessing file IDs limited to that VS

– rebalance users among NIC cards– move virtual IP addresses around dynamically

Virtual Servers

INDUSTRY

CONFERENCE

• 94 MB/sec read– single stream read, 9K MTU

• 99 MB/sec write– single stream write, 9K MTU

• All files much larger than cache– real I/O scheduling was occurring

Performance – single stream

INDUSTRY

CONFERENCE

Benefits

• Scale single export tree to high capacity– both in terms of gigabytes

– and ops/second

• Keep server utilization high– create VFSes wherever space exists

– independent of where data located in name space

• Use expensive class of storage– only when needed

– anywhere in the global name space

INDUSTRY

CONFERENCE

Benefits

• Use third-party or SAN storage– Spinnaker sells storage

– but will support LSI storage, others

• Kerberos and virtual servers– independent security mechanisms

– cryptographic authentication and

– IP address-based security as well

INDUSTRY

CONFERENCE

• Free data from its physical constraints– data can move anywhere desired within a cluster

• VFS move– move data between servers online

• VFS mirroring– Mirror snapshots between servers

• High availability configuration– multiple heads supporting shared disks

Near Term Roadmap

INDUSTRY

CONFERENCE

VFS Movement

• VFSes move between servers– balance server cycle or disk space usage

– allows servers to be easily decommissioned

• Move performed online– NFS and CIFS lock/open state preserved

– Clients see no changes at all

INDUSTRY

CONFERENCE

VFS Move

SpinServer cluster

Windowsdesktop

SpinServer1

SpinServer2

Q1 eng

Q2 acct

INDUSTRY

CONFERENCE

• Multiple identical copies of VFS– version number based

– provides efficient update after mirror broken

– thousands of snapshots possible

– similar to AFS replication or NetApp’s SnapMirror

INDUSTRY

CONFERENCE

Failover Pools

• Failover based upon storage pools– upon server failure, peer takes over pool

– each pool can failover to different server

– don’t need 100% extra capacity for failover

INDUSTRY

CONFERENCE

Failover Configuration

SpinServer 1

SpinServer 2

SpinServer 3

INDUSTRY

CONFERENCE

Additional Benefits

• Higher system utilization– by moving data to under-utilized servers

• Decommission old systems– by moving storage and IP addresses away

– without impacting users

• Change storage classes dynamically– move data to cheaper storage pools when possible

• Inexpensive redundant systems– don’t need 100% spare capacity

INDUSTRY

CONFERENCE

• Caching– helps in MAN / WAN environments

– provide high read bandwidth to single file

• Fibrechannel as access protocol– simple, well-understood client protocol stack

• NFS V4

Extended Roadmap

INDUSTRY

CONFERENCE

Summary

• Spinnaker’s view of NAS storage– network of storage servers

– accessible from any point

– with data flowing throughout system– with mirrors and caches as desired

– optimizing various changing constraints

– transparently to users

INDUSTRY

CONFERENCE

ThankYou

Mike KazarCTO

INDUSTRY

CONFERENCE

Design Rationale

• Why integrate move with server?– VFS move must move open/lock state

– Move must integrate with snapshot

– Final transition requires careful locking at source and destination servers

INDUSTRY

CONFERENCE

Design Rationale

• Why not stripe VFSes across servers?– Distributed locking is very complex

– and very hard to make fast

– enterprise loads have poor server locality

– as opposed to supercomputer large file patterns

– Failure isolation

– limit impact of serious crashes

– partial restores difficult on stripe loss

INDUSTRY

CONFERENCE

Design Rationale

• VFSes vs. many small partitions– can overbook disk utilization

– if 5% of users need 2X storage in 24 hours– can double everyone’s storage, or

– can pool 100 users in an SP with 5% free

spinnaker networks, inc

Documents

vecima networks inc

extreme networks, inc. (enterasys networks, inc

diy spinnaker sock

blue line spinnaker series - ullman...

spinnaker chadev

grandstream networks, inc. · grandstream networks, inc....

pascalliedcdnep.azureedge.net · spinnaker industries inc....

arista networks, inc. 2015 annual report€¦ · arista...

kenzan spinnaker meetup

spinnaker july '12 - issue 29

spinnaker vldb 2011

soc simulator on fpga using bluespec system...

spinnaker house - scrub island -...

orion telecom networks inc. · orion telecom networks inc....

juniper networks, inc.juniper networks, inc

spinnaker italia

sonus networks, inc

spinnaker portfolio sample

© spinnaker labs, inc. chubby. © spinnaker labs, inc. what...

f5 networks, inc