Parallel File System with High Performance

Applications

OCTOBER 21ST , 2013

REX TANAKIT VICE PRESIDENT OF SYSTEMS ENGINEERING

COMPANY CONFIDENTIAL 2

AGENDA

What is a Parallel File System?

Benefits of Parallel File System

SSD (Solid State Device)

Applications Performance

Benefits of Panasas Appliance

• Ease of use

• High performance

COMPANY CONFIDENTIAL 3

WHAT IS A PARALLEL FILE SYSTEM?

Simply put, it’s a file system that can scale as the number of

client/access grows.

Data

In the past

BIG

Today

COMPANY CONFIDENTIAL 4

PARALLEL FILE SYSTEM

BIG

COMPANY CONFIDENTIAL 5

PARALLEL FILE SYSTEM

Proverb: How do you eat an elephant?

Another answer:

- Get more people to help you eat it

- One “byte” at a time

Potential issues:

How to avoid bottlenecks?

Data integrity when sharing

High performance

Scalability

BIG

COMPANY CONFIDENTIAL 6

WHO NEED PARALLEL FILE SYSTEM?

BIG

The Common Themes:

A. Very complex problems and simulations

B. Very large number of data (files) being used concurrently

C. Very large data files

D. Very large number of concurrent users

E. Any or all of the above

“Traditional” file storage cannot keep up with these stringent demands

COMPUTE CLIENTS

RAID RAID RAID RAID

File

server

CPUs = RAID ENGINES

COMPUTE CLIENTS

PANASAS DIRECTFLOW

PARALLEL + SCALABLE

PANASAS STORAGE

Bottleneck

COMPANY CONFIDENTIAL 7

PARALLEL FILE SYSTEM APPLIANCE

Leading Performance that’s Fully Parallel

• Bladed design allows capacity and performance to

scale linearly to 8PB* and 150GB/s*

Easy to Deploy, Use, and Manage

• Tightly integrated system

• Set up or grow capacity in under ten minutes

• Single, global namespace

High Reliability and Availability

• High redundancy in hardware and software

ActiveStor 14

10 shelves

* Test limit

COMPANY CONFIDENTIAL 8

ACTIVESTOR BLADE ARCHITECTURE

CPU, cache, network

Orchestrates system activity

Metadata services

Up to 83TB per 4U chassis

Scalable to over 8 petabytes

Up to 1.6GB/s per chassis

Easy to install, easy to manage

Low Total Cost of Ownership

CPU, cache, data storage

Enables parallel reads/writes

Advanced caching algorithms

10GbE networking

InfiniBand Router 2 option

for IB connectivity

830TB & 15GB/s per 40U rack

ActiveStor Appliance

Switch

Module

Full

Rack

Storage

Blade

Director

Blade

COMPANY CONFIDENTIAL 9

HARDWARE: APPLIANCE MODEL

COMPANY CONFIDENTIAL 10

TWO TYPES OF DATA

Metadata

• Data about data, e.g. name, file size, date, etc.

• Small and random transactions

• Response time is important

Data

• Large, sequential I/Os

• Bandwidth is important

Ideal design

• SSD for Metadata

• HDD for Data

COMPANY CONFIDENTIAL 11

STORAGE HIERARCHY

Cpu w

Registers

L1

L2

L3

Memory

Hard Disk

L1: 1-3 ns

L2: 3-10 ns

L3: 10-20 ns

Access Time

30-60 ns

10,000,000 ns

SSD

100,000 ns

Typical Size

L1: 64KB/core

L2: 256KB/core

L3: 2-20 MB/chip

4-96 GB per system

TB+

120 – 500 GB

SSD

Lower latency

Excellent small file access

No seek “penalty”

Low power

COMPANY CONFIDENTIAL 12

PANASAS BLADES

Multi-Core CPU

HDD x2)

Director Blade

Storage Blade

CPU

2 Memory

Channels

ActiveStor 14 SSD

Faster CPU

• Accelerate metadata and small file performance

• Automatic SSD/SATA tier eases setup and manageability

COMPANY CONFIDENTIAL 13

BENEFITS OF SSD

ActiveStor 14* compared with AS 11, 12

• Benchmarks: mdtest for metadata (4KB file size)

Metric AS14T

Speed-up

from AS12

AS14T

(SSD+HDD)

AS12

(HDD only)

AS11 (HDD only)

4KB File Creates/s 1.9x 4250 2200 1650

4KB File Deletes/s 3.7x 6150 1650 1550

4KB File Stats/s (Random) 5.1x 14150 2750 2650

4KB File Reads/s

(Sequential)

8.6x 14150 1650 1600

4KB File Reads/s (Random) 10.5x 14150 1350 1300

*Using “typical” configurations

COMPANY CONFIDENTIAL 14

MULTI-PROTOCOL SUPPORT

Linux/Unix

Servers

Windows

Workstations

HPC Linux

Clusters

ActiveStor 11 ActiveStor 12

PanFS™ Parallel File System

ActiveStor™ Hardware Platform

NFS DirectFlow® CIFS

High Speed 10Gb or InfiniBand Network

ActiveStor 14

COMPANY CONFIDENTIAL 15

Parallel I/O throughput:

multiple concurrent jobs

Hardware:

• 4 Panasas AS12 shelves

• 32 8-core compute nodes

Landmark

SeisSpace/ProMAX

generating geoseismic trace

data

Parallel I/O is done using

JavaSeis (Open-source)

LANDMARK’S SEISSPACE®/PROMAX®

0

1000

2000

3000

4000

5000

6000

(Serial I/O) 32 64 128 256

MB

/s

# Processes

Scaling Concurrent Jobs

COMPANY CONFIDENTIAL 16

Serial I/O: constant as

process count increases

Parallel I/O: performance

increases • > 3X serial I/O at 80 P for Write

• > 4X serial I/O at 80 P for Read

• Low np limited by # of clients

• High np limited by # of disks

Storage subsystem: • ActiveStor 12: 3 shelves, 27 SBs

• Infiniband (QDR, Connected-mode)

STAR-CCM+ I/O RESULTS (N-1 ACCESS)

Courtesy of CD-adapco, Inc

0

200

400

600

800

1000

1200

1400

16 32 48 64 80

WR

ITE

TH

RO

UG

HP

UT

(M

B/s

)

PROCESS COUNT

Write Rates (Lemans 17 million cell case)

Serial IO

Parallel IO

hig

he

r

bett

er

0

500

1000

1500

2000

2500

16 32 48 64 80RE

AD

TH

RO

UG

HP

UT

(M

B/s

)

PROCESS COUNT

Read Rates (Lemans 17 million cell case)

Serial IO

Parallel IO

COMPANY CONFIDENTIAL 17

LS-DYNA 6.1.1. Implicit

Cyl0p5e6 Benchmark

• Nested cylinders

• 500K solid elements in model

• Motion on top and load on

bottom for nonlinear implicit

calculation

1 node (12 cores) per job

Scaling 1-24 Jobs

• PanFS (2 shelves)

• NFS (2 servers)

• Local Disk (2TB per node)

LS-DYNA PERFORMANCE

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

1 2 4 8 12 16 20 24

Seco

nd

s

Jobs

Scaling Job Count

PanFS

Local

NFS

LS-DYNA 971: Comparison of PanFS, NFS, and Local FS

NOTE: PanFS

Advantage in

Total Time vs.

Local FS

COMPANY CONFIDENTIAL 18

LS-DYNA 6.1.1.

Implicit

Cyl1e6

• Same of cyl0p5e6,

but twice the size of

the calculation

− 920K Solid

Elements

− 1M Nodes

• Each job on 1 node, 6

cores

LS-DYNA PERFORMANCE

0

5000

10000

15000

20000

25000

30000

1 2 4 8 12 16 20 24

Seco

nd

s

Jobs

Scaling Job Count

PanFS

Local

Lo

we

r

bette

COMPANY CONFIDENTIAL 19

UNIVERSITY OF MINNESOTA

ActiveStor 14

(20x)

Cluster Nodes

(50x)

Infiniband

Fabric

IB Routers

(16x)

10GbE

Fabric

Bandwidth limited by available nodes.

0

5000

10000

15000

20000

25000

1 2 4 6 8 10 12 14 16 18

write

read

….

MiB

/s (

ba

se

2)

COMPANY CONFIDENTIAL 20

BENEFITS OF PANASAS APPLIANCE

Use the right technology for the right workload

• Users do not have to think about how to take advantage of technology. It happens

automatically.

Easy to Deploy

• Installation in hours, not days

• Automatic recognition of new storage makes scaling a breeze

Easy to Use

• Compute nodes see a single, unified name space

• Files are automatically load balanced for maximum performance

Easy to Manage

• Single management view via PanActive Manager GUI (or CLI)

• Snapshots and user quotas reduce administrator time

• SNMP for integration into third party management tools

Easy to Scale

ActiveStor 14