the evolution of file systems - snia · the file systems evolution over time additional file...

The Evolution of File Systems

Presenter: Thomas Rivera

Senior Technical Associate, Hitachi Data Systems

Author: Christian Bandulet Principal Engineer, Oracle

The Evolution of File Systems © 2012 Storage Networking Industry Association. All Rights Reserved. 2 2

SNIA Legal Notice

The material contained in this tutorial is copyrighted by the SNIA. Member companies and individuals may use this material in presentations and literature under the following conditions:

Any slide or slides used must be reproduced without modification The SNIA must be acknowledged as source of any material used in the body of any document containing material from these presentations.

This presentation is a project of the SNIA Education Committee. Neither the Author nor the Presenter is an attorney and nothing in this presentation is intended to be nor should be construed as legal advice or opinion. If you need legal advice or legal opinion please contact an attorney. The information presented herein represents the Author's personal opinion and current understanding of the issues involved. The Author, the Presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK


Abstract

The File Systems Evolution Over time additional file systems appeared focusing on specialized requirements such as:

data sharing, remote file access, distributed file access, parallel files access, HPC, archiving, security, etc.

Due to the dramatic growth of unstructured data, files as the basic units for data containers are morphing into file objects, providing more semantics and feature-rich capabilities for content processing This presentation will:

Categorize and explain the basic principles of currently available file system architectures (e.g. Local, Shared, SAN, Clustered, Network, Distributed, Parallel, etc. Explain technologies like Scale-Out NAS, NAS Aggregation, NAS Virtualization, NAS Clustering, Global Namespace, Parallel NFS Review new file system architectures being developed

The Evolution of File Systems © 2012 Storage Networking Industry Association. All Rights Reserved. 4

Related Tutorials

We hope you checked out SNIA Tutorial:

Using File Server Protocols for Block-based Storage Workloads

Check out SNIA Tutorial:

Understanding Enterprise NAS Wednesday, 10:45AM

Check out SNIA Tutorial:

pNFS and NFS V4.2

Wednesday, 4:10PM

Check out Windows Server 8 File Server Wednesday, 11:40AM


?

Why File Systems Have Evolved

Scale Megabytes → Petabytes

Requirements High availability Data sharing Remote access Performance Archiving others…

Time

(Not a strict timeline—new capabilities are generally incremental)

..... Network

File System

Cluster File

System

SAN File

System

Shared File

System

Local File

System

Parallel File

System

Object File

System

Distributed File

System


File System

User space

Where File Systems Live

Kernel space

mmap()

User Application and Libraries (ls, mv, rm, cp, ...)

Process Management

Memory Mgmt Scheduler IPC

Data Cache* Segmap Cache

Volume Manager

System Calls (open(), close(), read(), write(), ioctl(), mmap(), ...)

DMA

VFS

Device Drivers

Buffers

*can be bypassed by using

direct I/O

Machine dependent code

Hardware


Data Blocks

data block data block data block data block data block data block data block data block data block data block data block data block data block

Host

What File Systems Do (UNIX example)

direct 0

direct 1

direct 2

direct 3

direct 4

direct 5

direct 6

direct 7

direct 8

direct 9

single indirect

double indirect

triple indirect

File Owner

File Type Permissions

Last Access

Size

# of links

. . .

File attributes:

Inode

0 1 2 3 4

5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

File locators: (“inodes”)

Data locators: (pointers)

Data: (blocks)


A File System Taxonomy

Local File System

Shared File System

SAN File System

Cluster File System

Network File System

Distributed File System

Distributed Parallel

File System

File Systems


Local File System

File system is co-located in the server with application

Local file system

Application

File System


Local File System

Separate “islands” of data Limitation: no data sharing

Application

File System

Application Application Application

File System File System File System


One Way to Share Data Scale-Up

Vertical scaling


Shared Data

Another Way to Share Data Scale-Out

Horizontal Scaling ...

Storage Network

Shared Device: A multi-LUN device shared among clients

Each client has exclusive access to a dedicated LUN ≠

Shared Data: A physical device shared among clients

Clients access LUNs concurrently


Data Access with Shared/Global File System

Separate logical and physical placement Metadata server File access is a three-step transaction...

Step 1:Request access

Metadata Server Client

Step 2: Metadata delivery

MDS Client

Step 3: Data access

MDS Client Metadata Server

Metadata Server


Shared Data

Shared/Global File System Asymmetric (“SAN File System”)

One active metadata server Typically homogeneous (scaling limited by metadata server capacity) Inter-node distance limited by storage network capability

Storage Network

Client Network

Application Server Application Server Application Server Application Server Application Server

Application e.g. Web Server



Metadata Server (active)

Metadata Server (passive)

Data Server Data Server Data Server




Shared Data

Storage Network

Shared/Global File System Symmetric (“Cluster File System”)

Metadata server in each node Typically homogeneous (scaling limited by internal communication, e.g., distributed locking) Inter-node distance limited by storage network capability

Client Network

Application Server Application Server Application Server Application Server Application Server






Data Server Data Server Data Server


Application (e.g. Web Server)

Data Server Data Server





Network File Systems (aka Proxy File Systems)

Enables sharing of files located on a file server among one or more client computers using a network protocol

Local File System

Application File System


Client

File System Server


Client


Client


Client

Network Protocol*

* e.g. NFS, CIFS, AFP, WebDAV, FTP, HTTP, ...

Network File System


Data

Network File System “Stack” (Example: Sun’s NFS)

SCSI Port

Volume Mgr

SCSI Driver

SCSI HBA

File System Application

NFS Client

Ethernet NIC

TCP/IP

RPC/XDR

NFS Server

Ethernet NIC

TCP/IP

RPC/XDR

LAN

SAN


Consolidation eases Management

Administration

Cost

Compliance

Global file sharing and collaboration

Location consolidation and optimization

But: WAN performance is low compared to LAN/SAN performance

Application

NFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Data

Wide Area Network File Systems

SCSI Port

Volume Mgr

SCSI Driver

SCSI HBA

File System

NFS Server

Ethernet NIC

TCP/IP

RPC/XDR

WAN

SAN


Data

Application

NFS/CIFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Application

NFS/CIFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Application

NFS/CIFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Application

NFS/CIFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Application

NFS Client

Ethernet NIC

TCP/IP

RPC/XDR

Improving Wide Area File System Performance

Application-specific optimizations: email, document management, SQL, ... Protocol-specific optimizations: HTTP, NFS, CIFS, WebDAV, FTP, TCP/IP, ... Transport acceleration: TCP accelerators Intelligent caching: read-ahead, deferred write, coherency, ... Data compression: algorithms, file-aware differencing, data aggregation, I/O clustering, chunk based de-duplication, cross-protocol data reduction, ...

SCSI Port

Volume Mgr

SCSI Driver

SCSI HBA

File System

NFS Server

Ethernet NIC

TCP/IP

RPC/XDR

SAN

Compression Engine

Ethernet NIC

TCP/IP

Ethernet NIC

TCP/IP

Compression Engine

Ethernet NIC

TCP/IP

Ethernet NIC

TCP/IP

LAN WAN LAN


/c /b /a

Distributed File System (DFS)

A network file system with files distributed among multiple file servers Not a parallel file system


Client

File System Server

File System Server

File System Server

Network Protocol

Single File System

/

/a /b /c client view:


Client

File

Aggregation of Storage Servers RAIN + RAID

(aka Network RAID) Global Namespace

Distributed Parallel File System

Segments of files distributed across storage nodes Enables parallel I/O to individual files (aka file striping)

File Server

File Server

File Server

File Server

File Server

Client Client Client

Network Protocol


NAS Aggregation

In-Band Solution Sometimes called “NAS Router”

IP Network

NAS Router

Global Namespace

SAN

File Server

Data

SAN

File Server

Data

SAN

File Server

Data


Client Client Client Client

Metadata Server (MDS)

Global Namespace

File Server

NAS Virtualization - Out-of-Band

Individual files / file segments pinned to file servers Files can be distributed and/or

replicated for parallel access Files can be striped for intra-file parallel

access Clients must locate the right file server e.g. NFSv4.1 (pNFS), Microsoft’s DFS

distributed files

striped files

replicated files

IP Network

File_A File_G File_B File_D

File_F File_H File_C File_E

File_K_1 File_K_2 File_K_3 File_K_4

File_A’ File_B’’ File_C’ File_B’

File Server

File Server

File Server


Application Server

IP

In-Band NAS:

IP

Out-of-Band NAS: Application Server Application Server

SAN SAN

Data

NAS Appliance

Data

NAS Appliance with NFSv4.1

pNFS extensions

NAS Virtualization – NFS4.1 pNFS

Storage Protocols: Block: FCP, iSCSI, SRP, SAS File: NFSv4.1 Object: OSD

Data path decoupled from control and metadata path

Application Server

NFSv4 client

Application Server

NFSv4 client

Application Server

NFSv4 client

Application Server

NFSv4.1 client with pNFS

Application Server


Application Server



Toward “Storage Grids” via NAS

NFS

Clustered Data Services

CIFS

HTTP

FTP

WebDAV

Each file pinned to a single server...

IP

VIP

Addr

ess

NFS

CIFS

Data Services

Local Files System

Classic Filer

VIP

Add

ress

Clustered Data Services

Cluster (Parallel) File System

NFS

CIFS

HTTP

FTP

WebDAV

All nodes serve all files...

Two variants:

Client

Client

Client


NAS Cluster is effectively a storage cloud

Cloud: The New Grid

Clients

Storage Cloud

Clients

Clie

nts

Clients

File Server Fi

le S

erve

r File Server


Data Segmentation

Media production, eCAD, mCAD, Office docs

Media-archive, DAM, Broadcast,

Medical imaging, Media-Internet

Transactional systems, ERP, CRM

BI, Data warehousing, Scientific,

Transaction archive

Fixed Data Dynamic Data

Stru

ctur

ed

Uns

truc

ture

d


The New Reality of Data Segmentation

Media production, eCAD, mCAD, Office docs

Media-archive, DAM, Broadcast,

medical imaging, Media-Internet

Transactional systems, ERP, CRM

BI, data warehousing, scientific,

transaction archive

Fixed Data Dynamic Data

Stru

ctur

ed

Uns

truc

ture

d

Semi Structured*

*Semi-Structured Data contains dynamic meta-data defined by users and/or applications


Owner, permissions, type, last modification, ...

Traditional Files

Data

Metadata


Object ID

Data

Metadata

Attributes User/application defined

Policies e.g., Replication

Methods e.g., Encryption

Semi-Structured Data



Data Blocks

Object

Object

Object

Object

Object

Inode Name OID

Name OID

Name OID

Name OID

Name OID

The File Object Model

Store

Data OID

Retrieve

OID Data

User/application defined

e.g., Replication

e.g., Encryption


Object ID

Data

Metadata

Attributes

Policies

Methods

Object Object


Managing File Objects

File objects can be managed like records in a relational database with user data as Binary Large Objects (BLOBs)

Object ID

Data

Metadata

Attributes

Policies

Methods

Object ID

Data

Metadata

Attributes

Policies

Methods

Object ID

Data

Metadata

Attributes

Policies

Methods

Object ID

Data

Metadata

Attributes

Policies

Methods

Object ID

Data

Metadata

Attributes

Policies

Methods

Object ID

Data

Metadata

Attributes

Policies

Methods

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Database Schema


Managing File Objects (Cont.)

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Objec

t ID

Data

Metad

ata

Attrib

utes

Polic

ies

Metho

ds

Indexes constraints/relationships Object search Full text search Join operations Virtual views SQL-like requests Cursors


Data Serving Hierarchy 3 Levels of Abstraction

Application may interface with the storage subsystem in any of three layers:

Block – highest performance and very little meta data File – high performance and some metadata Object – medium performance and rich metadata

Many to One

Many to One

Data Server Platform

Application

Object

File

Block


Application Content Repositories

Combination of application, database, data services and pointers into external file system

Application specific

Data Blocks

Application A Policies

Search Engine

Indexes

Pointers

Application B Policies

Search Engine

Indexes

Pointers

Application C Policies

Search Engine

Indexes

Pointers

Application D Policies

Search Engine

Indexes

Pointers

File System Name OID

Name OID

Name OID

Name OID

Object

Object

Object

Object


Data Blocks

File-Based Content Repositories

Combination of files system, database and data services

Application-independent

Application A Application B Application C Application D

File System

Policies

Search Engine

Indexes

Pointers

Name OID

Name OID

Name OID

Name OID

Object

Object

Object

Object


• Availability through file replication

• Sacrifice performance • Locality of data • No RAID protection • Peer-to-peer • Storage grid • Mesh topology • Flat namespace • Geographically dispersed • Heterogeneous • Spontaneous federations

File Systems for Clouds

File Systems are morphing into distributed object-based content repositories Compute

Nodes

Storage Cloud

Compute Nodes

Com

pute

N

odes

Com

pute N

odes

File Server

File

Ser

ver

File Server

Object_F Object_B' Object_E''

Object_A Object_C' Object_F''

Object_B Object_A' Object_D''

Object_C Object_E' Object_B''

Object_D Object_F'

Object_A''

Object_E Object_D' Object_C''


Please send any questions or comments on this presentation to SNIA: [email protected]

Many thanks to the following individuals for their contributions to this tutorial.

- SNIA Education Committee

Christian Bandulet Craig Harmer Paul Massiglia Thomas Rivera Joseph White

Q&A / Feedback

the evolution of file systems - snia · the file systems evolution over time additional file...

Documents