osdc 2014: thomas schend - introduction to synnefo open source cloud stack
DESCRIPTION
This talk wants to introduce you to Synnefo, an open source, scalable and production ready cloud stack. It consists of a vm management layer which is google ganeti and is essentially a cluster manager and delivers an integrated management of compute, network and storage. It also runs on of the shelf hardware and delivers live migration without or with shared storage. Synnefo is the orchestration and presentation layer which talks to ganeti via an API. To the users it presents a simple web UI and also exposes an openstack compatible API for automation called cyclades. Also it is possible to do flexible, L2 isolated networking. Also it provides a “dropbox” like storage services called pithos. It features a sync client for different platforms and a web UI. Synnefo offers all the benefits of an amazon like cloud but is geared towards persistent virtual machines. Also it is perfect as a replacement of the traditional virtualization stack and is easy to setup and use.TRANSCRIPT
![Page 1: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/1.jpg)
04/10/2023 OSDC 20141
Introduction to Synnefoopen source cloud stackThomas Schend, Systems Architect, CompuGroup Medical AG
Open Source Data Center Conference 2014
08. – 10. April, Berlin
![Page 2: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/2.jpg)
04/10/2023 OSDC 20142
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 3: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/3.jpg)
OSDC 2014
successfully worldwide in 35 countries for approx. 385,000 customers
10.04.20233
= 19 countries with own offices = additional 16 countries with customers
2012 Guidance 2013Customers 385.000
Group sales € 451 Mio. € 458 – 463 Mio.
EBITDA € 105 Mio. € 97 – 100 Mio.
employees 3.700
Indicators
Best provider of IT solutions for higher quality and efficiency in health care
Worldwide leader for MIT (Medical Information Technology)
High profitability and an average of 30% growth p.a.
Objectives
![Page 4: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/4.jpg)
04/10/2023 OSDC 20144
Bio
Systems Architect, responsible for Infrastructure DesignOSS EnthusiastLPIC-2,MCITP,VCP,CCA,DCIE,…
Contact: [email protected]
![Page 5: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/5.jpg)
04/10/2023 OSDC 20145
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 6: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/6.jpg)
04/10/2023 OSDC 20146
What do we need from IaaS?
ReliableScalable (Backend and VMs)Easy to manageLong lived workloads, not all VMs are EC2 instances ;-)
![Page 7: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/7.jpg)
04/10/2023 OSDC 20147
“VMs are not cattle, they are pets”
Vangelis Koukis, Technical Lead, Synnefo
Quote
![Page 8: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/8.jpg)
04/10/2023 OSDC 20148
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 9: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/9.jpg)
04/10/2023 OSDC 20149
Synnefo
Open SourceProduction-quality IaaSmade for persistent VMscommodity hardware – No SANLarge-scale Installationssimple UI for the end-useruses Ganeti as a Cluster Manager
![Page 10: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/10.jpg)
04/10/2023 OSDC 201410
Synnefo History
Design started late 2010Production since July 2011~okeanos* public cloud running currently ~8700 VMs
*http://okeanos.grnet.gr
![Page 11: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/11.jpg)
04/10/2023 OSDC 201411
Synnefo Layers
Image provided by GRNET
![Page 12: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/12.jpg)
04/10/2023 OSDC 201412
Synnefo birdseye view
Image provided by GRNET
![Page 13: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/13.jpg)
04/10/2023 OSDC 201413
Synnefo
Build with Ganeti as a backendAstakos: Identity/Account servicesCyclades: Compute/Network/Image/Volume servicesPithos: File/Object Storage servicesnf-manage commandline toolBasic helpdesk toolsnf-burnin / snf-burnin-runkamaki commandline clientCode on github
![Page 14: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/14.jpg)
04/10/2023 OSDC 201414
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 15: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/15.jpg)
04/10/2023 OSDC 201415
Synnefo Architecture I
Image provided by GRNET
![Page 16: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/16.jpg)
04/10/2023 OSDC 201416
Synnefo features - Astakos
Identity serviceProvides DashboardTrack Resource usageProject ManagementMultiple Auth methods
Twitter / Google / LinkedIn / LDAP / Active-directory
![Page 17: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/17.jpg)
04/10/2023 OSDC 201417
Synnefo features - Cyclades
Network managementL3 Routed/L2 VLANS/L2 isolation with mac filternfdhcpd
Archipelago storage abstractionceph/NFS/Gluster backendvery fast, thin cloningSnapshots
![Page 18: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/18.jpg)
04/10/2023 OSDC 201418
Synnefo features - Pithos
Object storage serviceEfficient syncingHashing before uploadDeduplication in backendWeb UI written in GWTClients for Windows, Macos and IOSSharing public and private
![Page 19: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/19.jpg)
04/10/2023 OSDC 201419
Synnefo / Ganeti Communication
Image provided by GRNET
![Page 20: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/20.jpg)
04/10/2023 OSDC 201420
written in python / djangoapache/gunicornpostgres databasesnf-gnt-eventd / snf-progress-monrabbitmqsnf-dispatchersnf-vncauthproxybuild for wheezy, can run fully virtualizedspeaks openstack API but does not share any code
Synnefo Architecture II
![Page 21: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/21.jpg)
04/10/2023 OSDC 201422
Secure image deployment of untrusted imagesIsolated from the ganeti nodeSupports Linux,Windows and BSD
customization tasksAssign hostnameDelete SSH keysChange passwordResize FilesystemRun unattended install (windows)Inject files
Synnefo - snf-image
![Page 22: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/22.jpg)
04/10/2023 Central IT Services23
Synnefo snf-image deployment process
gnt-node evacuate -I hail node3
Imag
e pr
ovid
ed b
y G
RNET
![Page 23: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/23.jpg)
04/10/2023 OSDC 201424
Agenda
IntroductionWhat we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 24: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/24.jpg)
04/10/2023 OSDC 201425
Ganeti History
Very mature, production readyopensourced since August 30, 2007 by googleuses Mature open source tools LVM,DRBD,python,haskellDifferent hypervisors (Xen,KVM,LXC)Low overhead for cluster management
![Page 25: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/25.jpg)
04/10/2023 OSDC 201426
Ganeti Usage at google
Usage at googlesmall Clusters in Offices (OIB)large Clusters in Datacenters for virtual desktops
mostly using XENNot used for user facing services
![Page 26: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/26.jpg)
04/10/2023 Central IT Services27
Ganeti features I
Scale from 1-200 nodes and beyondLive migration without shared storageNoderoles: master,master candidate,regular nodesNodegroupsOVF import toolBackup with included toolJob queueCommands start with gnt-
![Page 27: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/27.jpg)
04/10/2023 OSDC 201428
Ganeti features II
OS Interface to provision new instancessnf-image secure image deploymentganeti-instance-debootstrap, image, ganeti-os-defsHtools to balance clusters, plan capacity and do maintenance
supported file,plain,sharedfile,ceph,NFS,Gluster Ext storage provider with plugins for Archipelago, HP EVA,,IBM Storwize
![Page 28: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/28.jpg)
04/10/2023 OSDC 201429
Ganeti advanced features
Hooks for every action (pre/post)Tags for every objectExclusion tags to keep machines apart from each otherDeploy flash in every node for high performance (flashcache,bcache,dm-cache)OOB management interfaceGaneti watcher
![Page 29: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/29.jpg)
04/10/2023 OSDC 201430
Ganeti advantages
No need for special hardware (SAN)But SAN/Shared Storage is supportedManage cluster as an entity not single nodes (libvirt)Few dependenciesNo databaseEasy recovery in case of hardware outageNo SPOF which can affect the reliability of VMsVery good documentation
![Page 30: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/30.jpg)
04/10/2023 OSDC 201431
Deployment automation puppet,chef,cfengine,..Monitoring icinga/nagios/check_mk/…Openvswitch LACP + 10GBeAutorepairPackages for all major OS debian,ubuntu,rhel/centos
Ganeti best practices
![Page 31: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/31.jpg)
04/10/2023 OSDC 201432
Ganeti management interfaces
cli with bash completionLUXIRAPI
Different web UIs using the RAPIGaneti web managerGanetimgrSynnefo
![Page 32: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/32.jpg)
04/10/2023 OSDC 201433
Ganeti quick command reference
gnt-cluster init / info / verify / command / copyfile
gnt-node add / list / failover / evacuate
gnt-instance add / info / replace-disks
gnt-job list …
gnt-network …
gnt-backup …
…
![Page 33: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/33.jpg)
04/10/2023 OSDC 201434
Ganeti htools - allocation and placement
written in haskell
hail
hbal
hspace
harep
hinfo
hroller
…
![Page 34: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/34.jpg)
04/10/2023 OSDC 201435
Typical Ganeti Clsuter
Node01 Node02 Node03
gnt-cluster init [-s ip] --enabled-hypervisors=kvm clustergnt-node add [-s ip] node2gnt-node add [-s ip] node3gnt-cluster verifygnt-instance add -t drbd -I hail -o debootstrap+default vm
Veea
m v
irtua
lizati
on st
enci
ls
![Page 35: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/35.jpg)
04/10/2023 OSDC 201436
Ganeti Failover and Repair
gnt-node modify -O yes node01gnt-node failover --ignore-consistency node01gnt-node evacuate -I hail node01
Node01 Node02 Node03
Veea
m v
irtua
lizati
on st
enci
ls
![Page 36: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/36.jpg)
04/10/2023 OSDC 201437
“Ganeti didn't have any generally known meaning before this project. And now it means kick-ass cluster management”
Martin Beauchamp, on the Ganeti Users List
Quote
![Page 37: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/37.jpg)
04/10/2023 OSDC 201438
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 38: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/38.jpg)
04/10/2023 OSDC 201439
Login to Astakossyncing and Pithos UI and AppCreate VMs with Cyclades UISnapshot FeatureOOB ConsoleHotplug of NICs and NetworksFloating IPsClean-up
Demo
![Page 39: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/39.jpg)
04/10/2023 OSDC 201440
Agenda
IntroductionWhat do we need from IaaS?Introducing to SynnefoComponentsGanetiDemoQ & A
![Page 40: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/40.jpg)
04/10/2023 OSDC 201441
http://www.synnefo.org
https://github.com/grnet/synnefo
https://code.google.com/p/ganeti/
Try it!
![Page 41: OSDC 2014: Thomas Schend - Introduction to Synnefo open source cloud stack](https://reader033.vdocuments.site/reader033/viewer/2022061214/549a56cdb4795979508b4694/html5/thumbnails/41.jpg)
04/10/2023 OSDC 201442
Q & A