a petabyte cloud the simple way - fosdem · network virtual networks network topology o security...

17

a petabyte cloud the simple way +

Upload: others

Post on 29-Jul-2020

3 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

a petabyte cloud the simple way

+

Page 2: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Simon Haly@simon_haly

Lorenzo Faleschini@penzoiders

Page 3: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 4: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

++ countless hours of integration and running in production

=

Page 5: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 6: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 7: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 8: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 9: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

brought you by

https://github.com/cloudweavers/lizardfs-tm_mad

grab it while it’s hot on

Page 10: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 11: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 12: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 13: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

key features● KVM + OpenNebula + LizarFS production environment in 5 minutes● Self managed, Self healing, Self tuning● Survies ANYTHING!● SDN capabilities● VDI support ● Backup, DR tools out of the box● Scales up to 25 nodes with no special config needed (30 mins to PB cluster!)● Stupid low price point

Page 14: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

Page 15: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

SYSTEM ELEMENTS

Skytechnology sp. z o.o., ul. Miłobędzka 35, 02-634 Warszawa, www.skytechnology.pl

Page 16: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

COMMUNICATION WITHIN SYSTEM

Skytechnology sp. z o.o., ul. Miłobędzka 35, 02-634 Warszawa, www.skytechnology.pl

Page 17: a petabyte cloud the simple way - FOSDEM · Network Virtual Networks Network Topology O Security Groups Infrastructure Hosts Distributed Storage Dashboard VMs 27 CPU hours Hosts 2

AUTO BALANCING

Skytechnology sp. z o.o., ul. Miłobędzka 35, 02-634 Warszawa, www.skytechnology.pl

Red Hat Satellite 6 · Discovered Hosts - a list of bare-metal hosts detected on the provisioning network by the Discovery plug-in. Content Hosts - a list of hosts that manage tasks

Petabyte Scale Data at Facebook - Borthakur

Chapter 3 Transport Layer - University of Washingtoncourses.washington.edu/ee565/handouts/chapter3.pdfTransport vs. network layer network layer: logical communication between hosts

Petabyte-scale computing for LHC

On Measuring the Similarity of Network Hosts: Pitfalls

The Personal Petabyte The Enterprise Exabyte

2010 AIRI Petabyte Challenge - View From The Trenches

A Escola na Era do PetaByte

Introduction Widespread unstructured P2P network Currently between 200,000 & 300,000 hosts Currently between 200,000 & 300,000 hosts Ideal as a research

Surviving the Petabyte Age: A Practitioner's Guide

Module 6: Designing Security for Network Hosts. Overview Creating a Security Plan for Network Hosts Creating a Design for the Security of Network Hosts

Oscon 2015 - OpenIO: Enabling the petabyte-sized mailboxes

Exceptional Data Insight for Petabyte- Scale Unstructured

Review of Network Technologies. Basic of Network concepts Computer Network – A Communication system for connecting computers/hosts Why? – Better connectivity

Chapter 6 Network Communications and Protocolssjkuyath/ETEE3281/ch06.pdf•65,534 hosts per network address –Class C: first octet between 192-223 •254 hosts per network address

Lab 2.4.4 VLSMkowon.dongseo.ac.kr/~hjlee/CCNP5v2_student_manual.pdf · 2015-02-06 · Lab 2.4.4 VLSM 28 hosts 60 hosts 12 hosts 12 hosts Network: 192.168.10.0 Objectives: • Create

Introduction1-1 Network Overview. Introduction1-2 A closer look at network structure: network edge: applications and hosts network core: m routers

Computer Network Basics.pdf - RUFORUM - …ruforum.org/sites/default/files/Computer Network Basics.pdfComponents of a computer network A computer network is composed of: Hosts (PCs,

THE PETABYTE CHALLENGE: 2011 IOUG DATABASE GROWTH SURVEY

Network Appliance Confidential Fibre Channel Supported Configurations Switching & Hosts

Netflix: Integrating Spark At Petabyte Scale

Tera/Petabyte data distribution architectures

Live Migration of an Entire Network (and its Hosts)

FILE NO: TCT/MCA… · Web viewBridges add a level of intelligence to the network by using the MAC address to build a table of hosts, mapping these hosts to a network segment and

Transferring a Petabyte in a Day - Argonne National Laboratorykettimut/publications/fgcs18-1pb.pdf · Transferring a Petabyte in a Day1 Rajkumar Kettimuthua, Zhengchun Liua,, David

Petabyte-Scale Text Processing with Spark

Lab 2.4.4 VLSM - Dongseodongseo.ac.kr/~hjlee/CCNP5v2_student_manual.pdf · · 2015-02-06Lab 2.4.4 VLSM 28 hosts 60 hosts 12 hosts 12 hosts Network: 192.168.10.0 ... When construction

Transport Layer - Tulane Universityzzheng3/teaching/cmps6750... · Transport vs. network layer §network layer:logical communication between hosts §transport layer:logical communication

BBM: Bayesian Browsing Model from Petabyte -scale Data

Petabyte scale on commodity infrastructure

Building PetaByte Servers

· Web view172.16.255.255, which is all subnets and hosts on network 17.16.0.0 10.255.255.255, which broadcasts to all subnets and hosts on Network 10.0.0.0. The Hierarchical IP Addressing

Konsep Subnetting · Web view16. The network address of 172.16.0.0/19 provides how many subnets and hosts? a. 7 subnets, 30 hosts each b. 7 subnets, 2046 hosts each c. 7 subnets, 8190

Transportation Layer (1). Transportation Layer Very similar to the data link layer. – two hosts connected by a link or two hosts connected by a network

PetaByte Storage Facility at RHIC