self-service virtual environments at deutsche telekom ... · self-service virtual environments at...
TRANSCRIPT
Self-service virtual environments at DeutscheTelekom based on OpenStack
Alexander StellwagDeutsche Telekom AGProducts & Innovation
Agenda
● Introduction● Short History Of Wolke-7● Planning and Setup● Virtual Private Environments● Version 2: havana and neutron● Lessons Learned● Q&A
Introduction
● Alex Stellwag● 43 Years● Started at Deutsche Telekom
in 2001● Cloud Architect since 2011● OpenStack since „Diablo“
Short History of Wolke-7● Started back in late 2011 after major reorganization to gain knowledge of
OpenStack● Dec 2012: initiated two basic use cases
– Plain IaaS, configurable via EC2-API
– Fully managed environments (PaaS)
● Both available from one cloud environment● Internal customers only, no public services● Core principles
– Stay open source
– Build over partnering
Intranet Infrastructure
● Supermicro servers (2UTwin² and 2U 12 Disk)● Ubuntu 12.04 LTS (Kernel 3.2 and 3.11)● OpenStack packages from UCA● Ceph packages from Inktank● I/S automation via FAI1 and puppet● Monitoring via nagios / check_mk
1 http://fai-project.org
Some Figures
● 12 compute nodes with 2x Xeon X5650 (6 cores, 2.67 GHz, Intel HT enabled), 96 GB Ram, 2x 250 GB SSD each
● 3 Ceph nodes with 2x Xeon E5-2630, 64 GB Ram, 2x 240 GB SSD + 10x 2TB S-ATA (OSDs) each. ==> ~60 TB net
● 2x Cisco Nexus 5548 for tenant VLANs and floating IPs● 2x Cisco 48 port 1 GE for administration
Planning and Setup
● Setting up OpenStack is quite painless thanks to the official docs● We needed a complete DC
– Location– Connectivity
– Basic I/S services (DNS, Mail, Proxy)– Automation services (FAI, puppet)
● Complex Setup due to internal security requirements● Adapting the puppet modules for our needs took a while
Virtual Private Environments I
● Simple applications (apache, mysql, tomcat)– Provided by user-data
– Published on project website
– User has to copy & paste them into dashboard
– Designed to fulfill security requirements
– Full self-service via OpenStack dashboard
– Users have full control and responsibility
#cloud-config
# this cloud config code installs apache2-worker webserver# and changes several configuration directives to already meet# some GIS requirements# after customization of the webserver the requirements have to be# checked agein
# supported OS: Ubuntu 12.04
# addtional packages to be installedpackages: - apache2 - libapache-mod-security
# commands to modify the default installationruncmd: - a2dismod cgid # req 7 - sed -i 's/\(.*Alias \/\(cgi-bin\|doc\)\)/#\1/; //{s/^/#/}; //,/Options /{s/\(.*\n\t\t\tdeny from all\n\t\t \ <\/LimitExcept>/}' /etc/apache2/sites-available/default # req 9,11,12,13,16,18 - sed -i 's/^/#/' /etc/apache2/mods-available/alias.conf # req 16 - sed -i 's/^\s*ServerTokens .*/ServerTokens Prod/' /etc/apache2/conf.d/security # req 19 - service apache2 restart # restart webserver to apply changes
Virtual Private Environments II
Virtual Private Environments III
● Complex environments (currently CI/CD for Java only)– Jenkins, git, tomcat
– Set up via bash and python scripts
– Not yet integrated in dashboard, ordered via e-mail
– Fully managed by Test & Development team
Version 2: havana & neutron
● Started in 10/2013● New location, so again we needed to setup a complete DC● Ready for production in May 2014● Currently looking for dedicated operations team● Heat installed and enabled but Virtual Environments not ported
over yet.● Users may use their own stack templates
Infrastructure Changes
● No dedicated network node, all agents distributed amongst compute nodes
● Multiple L3 and DHCP agents per L3 network● Custom python script to check availability of agents and
reschedule if necessary● Works, but we have no long-time experience● Currently looking into Icehouse, but migration has issues
(mainly OS-wise)
New Hardware
● 11 compute nodes with 2x Xeon E5-2660 v2 (10 cores, 2.2 GHz, Intel HT enabled), 128 GB Ram, 2x 400 GB SSD each
● 8 Ceph nodes with 2x Xeon E5-2620 v2 (6 cores, 2.1 GHz, Intel HT enabled), 32 GB Ram, 2x 100 GB SSD + 10x 4TB S-ATA (OSDs) each. ==> ~312 TB net
● 2x Alcatel Lucent OS 6900-T40 for VxLAN tunnels and floating IPs
● 2x Alcatel Lucent OS 6850 for administration
Lessons Learned
● I/S deployment is hard – even if you start on a green field● Deploying OpenStack without the ability to fix the code is
difficult (and sometimes frustrating)● HA for neutron is a complex beast● Deploying multi-tier applications without heat is no fun
Lessons Learned II
● Manage internal stakeholder for proper funding● Many customers don't know the difference between
virtualization and cloud computing● Change management program needed because cloud impacts
95% of operations roles● For large and inhomogeneous teams SCRUM is not the right
project management method