openstack & mysql
Post on 12-Apr-2017
693 Views
Preview:
TRANSCRIPT
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack & MySQL
Airton Lastori airton.lastori@oracle.com
Copyright © 2015, Oracle and/or its affiliates. All rights reserved.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
An Introduction to OpenStack
An Introduction to Trove
Why MySQL Enterprise Edition?
Integrating MySQL Enterprise with Trove
What’s Coming Next?
1
2
3
4
5
4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
An Introduction to OpenStack
5
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The OpenStack Mission: to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Contribuidores
• Oracle
• HP
• Microsoft
• IBM
• Redhat
• AT&T
• Muitos outros...
Usuários BR
• Instituto Federal de Santa Catarina
• PUCRS
• UFMG
• Globo.com
• R7
• Talvez mais...
7
Fundação Openstack.org
www.openstack.org/user-stories www.openstack.org/foundation/companies
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Momentum • Launched in 2010, OpenStack has grown to become one of the largest open source
project ever created. Some numbers from http://www.stackalytics.com/ as of May 2015:
– Over 400 companies contributed code to OpenStack (Kilo)
– Almost 2700 individuals contributed code to the project (Kilo)
8
0
20
40
60
80
100
120
140
Companies Contributing Up to Juno
0
200
400
600
800
1000
1200
1400
1600
Individuals Contributing Up to Juno
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Momentum
Supporting Companies: http://www.openstack.org/foundation/companies/
9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Top 5 Business Drivers for OpenStack
Source: http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014
10
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Deployments
Source: http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014
11
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Workloads & Databases
Source: http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014
12
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What is OpenStack?
• OpenStack is an open source cloud operating system designed to control large scale cloud deployments
• OpenStack controls compute, storage, network, identity management, orchestration, and much more … pool of resources ready-to-use
13
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 14
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Cloud Privada
• On Premise
– Distros & Appliances
– Instalação “na unha”
• Hosted Private Clouds
– Rackspace
– Ubuntu
Outros...
Cloud Pública
• Rackspace
• HP
• Vexxhost
• Ultimum
• KIO
• Data Centred
Vários outros...
15
Como usar
www.openstack.org/marketplace
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Distros & Appliances
• devstack.org
• Oracle (Linux e Solaris)
• Tesora
• HP
• Redhat
• VMWare
• Muito mais...
Guias de Instalação
• Oracle Linux, Redhat, CentOS 7
• Ubuntu 14.04 (LTS)
• openSUSE 13.2 e Enterprise SUSE
• Debian 8
16
Distros, Appliances ou Instalação “na unha”
docs.openstack.org www.openstack.org/marketplace/distros
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Cloud Operating System
17
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Services
• Compute Service : Nova
• Object Storage Service : Swift
• Block Storage Service : Cinder
• Image Service : Glance
• Identity Service : Keystone
• Network Service : Neutron
• Dashboard : Horizon
• Telemetry Service : Ceilometer
• Orchestration Service : Heat
• Database Service : Trove
• Data Processing : Sahara
• Bare-Metal Provisioning : Ironic
And many more …
18
core
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Most OpenStack Services Use MySQL Availability and Scalability of MySQL Is Crucial for Operation and Performance
• Very large transactions from services going into MySQL • To avoid bottleneck
• Automatically add nodes for scalability and availability • Instead of downtime to restructure
• Important to have back ups (MySQL Enterprise Backup) • If MySQL goes down your service and cloud is down
http://docs.openstack.org/openstack-ops/content/architecture.html
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack is About Abstraction and Integration
20
Block Storage service
Cinder
Network service
Neutron
Compute service
Nova Application Programming Interfaces
Plugins then Implement the API (partial list)
Common Interface – OpenStack API
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Amazon OpenStack Trove
21
How Does OpenStack Trove Compare to Amazon RDS …and DynamoDB …and RedShift
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Goals: OpenStack With Oracle Linux and Oracle VM
Interoperability
• Help make OpenStack a first class solution for enterprise customers
• Work with the OpenStack community to support enterprise applications in an OpenStack cloud
• Provide choice for managing hypervisors with Oracle Linux and Oracle VM
• Ensure Oracle Linux and Oracle VM can work with any OpenStack solution customers choose
• Work with third party vendors to offer joint solutions
22
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Across Oracle’s Portfolio
Horizon
Cloud Management
Oracle Linux , Oracle Solaris, Oracle VM,
Nova Compute Virtualization
Oracle Linux , Oracle Solaris, Oracle Virtual
Networking
Neutron Cloud Networking
Oracle Linux, Oracle Solaris, Oracle ZS3,
Oracle Axiom, Oracle StorageTek Tape,
Oracle FS1
Cinder/Swift Cloud Storage
Oracle Linux, Oracle Solaris, Oracle VM
Templates
Glance Image Deployment
23
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
End to End Full Stack Support
24
Control node Compute node n Compute node 1
VM VM
VM
VM VM
VM
Clusterware
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle OpenStack High Availability MySQL Cluster Active/Active
MySQL Cluster RabbitMQ Keepalived Nova Neutron Memcached
Cinder Swift Keystone Glance Heat Horizon
Docker Containers
Controller Node(s)
MySQL Cluster RabbitMQ Keepalived Nova Neutron Memcached
Cinder Swift Keystone Glance Heat Horizon
Docker Containers
Container life cycle management (Ansible)
Management Controller Nodes
API
MySQL Cluster Data Nodes
Management
Data Layer
HA
Pro
xy
Current A/A: Galera
MySQL Cluster
Scaling Limited write Linear write
Performance Standard Real-time
Online DDL No Yes
Auto Sharding No Yes
NoSQL APIs No Yes
Load Balancing No Yes
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
An Introduction to Trove
26
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Current State of Database Deployments
• Ever Growing Database Population
• Multiple versions and patch levels
• Compliance challenges
• Poor resource utilization
• High cost of deployment and operation
Siloed, Dispersed, Varied and Complex
27
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
DBaaS Key Capabilities
Self Service Metering
Elasticity
Service Catalog High Availability
Security
End-users can provision, monitor & manage for greater agility
Allows chargeback for cost allocation
Support dynamic workloads with minimal excess capacity
Standardized database services for rapid, error-free deployment
Avoiding costly unplanned downtime
Protecting against external attacks, misuse of information & ensuring regulatory compliance
28
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Highlights:
• Scalable and reliable cloud DBaaS provisioning
• Built for needs of Cloud users and DBAs
• Provision/manage multiple MySQL EE instances
• Automates administration across lifecycle:
– Deployment
– Configuration
– Patching
– Backups / restores
– Monitoring
Benefits:
• Reduces database sprawl
• Supports rapid provisioning
• Enhances security
• Improves performance and availability
• Automates management
Improves agility, security, and reduces TCO
29
DBaaS with OpenStack Trove and MySQL Enterprise
* MarketsandMarkets forecasts the Cloud Database and DBaaS Market to grow from $1.07 billion in 2014 to $14.05 billion by 2019, at a Compound Annual Growth Rate (CAGR) of 67.30% during the forecast period of 2014-2019.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Tuning
• Automatically tune my.cnf
• InnoDB Buffer Pool size
• Replication options
• Max Connections
• …
• Sane defaults
• InnoDB only
• Disable LOAD DATA INFILE
• Disable SELET INTO OUTFILE
• API to programmatically set configuration groups
Management
• Create database / schema
• Create users
• Grant permissions
• Enable root user
• Resize flavor
• Resize volume
• Full and incremental backups
• Create replicas
• Create clusters
30
Security
• Security groups
• Turn off SSH
• Remove anonymous user(s)
• Remove non-localhost user(s)
• Remove local file access
• Mangle root user password
• Apply security patches automatically
Trove is more than just provisioning…
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Trove Architecture
31
• Built on OpenStack – Backups to Swift
– Images in Glance
– Nova management
– Data in Cinder
• Major components – API
– Taskmanager
– Conductor
– Guest Agent
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Provisioning Models
32
PDB
Pre-provisioned
hardware
Oracle 11g
Pre-provisioned
hardware
Mongo DB
Pre-provisioned
hardware
MySQL
OpenStack Cloud
Oracle 12c Container Database
PDB PDB
Trove DBaaS
Trove Instance
Trove Instance
Oracle Trove
Instance
MySQL
Cluster
Nova Instance
Oracle
Nova Instance
Oracle
Trove Instance
MySQL
Cluster
Single Instance
Provisioning
Cluster
Provisioning Multi-tenant
DBMS Provisioning
Provision from
Inventory
PDB
PDB
* Oracle Database 11g and 12c are currently only supported in Tesora DBaaS Enterprise
*
*
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Enterprise Hardened Trove: Tesora DBaaS Platform
OpenStack Trove Community Edition
Enterprise DBaaS Features
Early Access to Upstream
24/7 Support
Guest Images Proprietary DBs
Simplified Install/Config
Community Code
Enterprise Edition
Simplified Install/Config
Extensive Testing/Bug Fixes
Guest Images O/S DBs
Community Code Community Code
Extensive Testing/Bug Fixes
Guest Images O/S DBs
Enhanced GUI Management
33
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• As a Trove user
– OpenStack distribution that includes Trove (e.g. Oracle OpenStack) • http://www.oracle.com/us/technologies/linux/op
enstack/overview/index.html
– Tesora DBaaS platform, a standalone Trove-based DBaaS product • http://tesora.com/download-tesora-dbaas-
platform-community-edition
• As a Trove developer $ git clone http://github.com/openstack/trove-integration
$ cd trove-integration/scripts
$ ./redstack install
$ ./redstack kick-start mysql
• On top of DevStack
– Add to localrc: • ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond
– Swift should also be enabled for Backup and Restore. • ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account
• SWIFT_HASH=<swift-hash-here>
34
Getting Started with OpenStack Trove
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Why MySQL Enterprise?
35
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Why MySQL Enterprise Edition? In Addition to all the MySQL Features you Love
Insure Your Deployments
Get the Best Results
Delight Customers
Improve Performance & Scalability
Enhance Agility & Productivity
Reduce TCO
Mitigate Risks
Get Immediate
Help if/when Needed
Increase Customer
Satisfaction
36
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Management Tools Advanced Features Support
• Scalability • High Availability • Security • Audit • Encryption
•Monitoring • Backup • Development • Administration •Migration
• Technical Support • Consultative Support •Oracle Certifications
MySQL Enterprise Edition
37
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL HA in OpenStack
• High Availability
– Server monitoring with auto-promotion and transparent application failover
• Optionally scale-out through sharding
– Application provides shard key
– Tools for resharding
– Global updates & tables
• Connection options
– Fabric-aware connectors
– MySQL Router
• Server provisioning using OpenStack
– Supports the Nova and Neutron APIs
MySQL Enterprise High Availability + Sharding-Based Scale-out
Router
Application
Read-slaves
SQL
HA group
Connector
Application
38
Group Replication cluster
Fabric node cluster
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Active/Active Update Anywhere
– Conflict detection and resolution (transaction rollback)
– Optimistic State Machine Replication
• Automatic group membership management and failure detection
– No need for server fail-over
– Elastic scale out/in
– No single point of failure
– Automatic reconfiguration
• Well integrated
– InnoDB
– GTID-based replication
– PERFORMANCE_SCHEMA
– Row based replication
– Multi-threaded slaves
MySQL Group Replication
Application
MySQL Nodes Replication
Plugin
API
MySQL Server
Group Comms
labs.mysql.com
39
For Active/Active OpenStack Setups
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Connection and Transaction routing
• Transparently improve your MySQL apps
– Transparent MySQL Fabric support • Transparent HA
• Transparent Sharding
– Transparent support for MySQL Group Replication clusters
– Transparent support for custom clusters and HA setups
• Easily extendable using plugin APIs
• Many new plugins to come
– Aggregation, Binary Log, Intelligent Load Balancing, Read/Write splitting, …
40
MySQL Router Transparent Routing for Complex MySQL Installations
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Scaling, Security, and Data Protection
MySQL Enterprise Scalability
MySQL Enterprise Monitor
MySQL Enterprise Backup
MySQL Enterprise Security
MySQL Enterprise Encryption
MySQL Enterprise Audit
MySQL Enterprise Authentication
MySQL Enterprise High Availability
Oracle Enterprise Manager for MySQL
41
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Integrating MySQL Enterprise & Trove
42
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise & Trove
43
• Create a MySQL Enterprise machine image for use with Trove
– Pull a standard image from Glance and add/install: • MySQL Enterprise Server
• Default MySQL config (/etc/mysql/my.cnf) that enables all Enterprise Plugins
– Can later use a Trove configuration group to sync and manage options across all instances
• MySQL Enterprise Backup
• MySQL Enterprise Monitor Agent
– Configured to know how to reach out to the MySQL Enterprise Monitor Server
• Trove Guest Agent
– Configured to know how to reach the Trove Server (API, Taskmanager, Conductor)
– Customized backup strategy definitions for MySQL Enterprise Backup
• trove/guestagent/strategies/backup/mysql_impl.py
• See: http://dev.mysql.com/doc/mysql-enterprise-backup/3.12/en/mysqlbackup.usage.html
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Store customized MySQL Enterprise Trove image in Glance
– Or skip all of the image related steps • Official certified Oracle Linux + MySQL Enterprise Trove images
• Tesora DBaaS Enterprise Certified MySQL Enterprise Trove images
• Create necessary entries in Keystone • User, role, tenant, service, endpoint
• Create MySQL datastore within Trove
– Use trove-manage client • Sync Trove information to OpenStack’s MySQL data store (trove-manage
db_sync)
• Create MySQL datastore using new Trove machine image in Glance
MySQL Enterprise & Trove
44
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Configure MySQL Enterprise Backups
– Specify MySQL Enterprise Backup as tool for backups (/etc/trove/trove.conf):
[mysql]
backup_strategy=mysqlbackup
backup_incremental_strategy={'mysqlbackup': 'incremental'}
• Instances can then be created with trove client or via Horizon web UI
MySQL Enterprise & Trove
45
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise & Trove
46
• Launch MySQL EE instance
– Flavor from Nova
– Cinder volume for the datadir
• Initialize the instance – Specify default accounts
– Specify default schemas
• Optionally restore from a backup
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 47
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
What’s Next?
48
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Plans for MySQL Enterprise
• MySQL 5.7 (GA)
– Improved scalability, performance, security, and ease-of-use
– Server Team Blog : http://mysqlserverteam.com
• MySQL Group Replication (Labs) – Easy HA with Active/Active clusters
– Replication & HA Team Blog : http://mysqlhighavailability.com
• MySQL Router (GA)
– Load balancing, transparent failover, aggregation, transparent sharding, … stay tuned!
• MySQL Fabric (GA) – Fabric 1.6 supports multi-node clusters
49
Making MySQL “Cloud Native”
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
The Future of MySQL Scaling (HA + Sharding)
Global Data Shard 1 Shard 2
MySQL Fabric controller cluster
SQL Queries
Server/Shard State & Mapping
Global Group HA Group
Coordination and Control
HA Group
Group Replication cluster
50
Group Replication cluster
Group Replication cluster
MySQL Router
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Plans for OpenStack
• Liberty (October 2015) and beyond …
– Queue Service (Zaqar)
– Shared File System (Manila)
– DNS Service (Designate)
– Key Management (Barbican)
– Container Service (Docker and Kubernetes: Magnum)
– Docker deployment and management tools (Kolla) • Used heavily in Oracle OpenStack for Oracle Linux R2
– Additional details: http://status.openstack.org/release/
• Next release is Mitaka (April 2016)
51
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Plans for Trove
• High Availability
– Auto-scaling up and down based on demand
– Failure detection and auto-failover
• Disaster Recovery – Cross region replication and failover
• Support for additional datastores
• Better integration with other OpenStack modules (e.g. Ceilometer)
• Improved security, monitoring, metering
• API for accessing datastore log files
– For example: Enterprise Audit log, Slow Query log, Error log, etc.
52
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Appendix
53
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
OpenStack Resources
• Docs
– http://docs.openstack.org/
• Wiki
– https://wiki.openstack.org/wiki/Main_Page
• Users Stories and Case Studies – http://www.openstack.org/user-stories/
• Community help
– https://wiki.openstack.org/wiki/IRC
– https://wiki.openstack.org/wiki/Mailing_Lists
54
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Trove Resources
• Docs
– http://docs.openstack.org/developer/trove/
• Wiki
– https://wiki.openstack.org/wiki/Trove
• Launchpad – https://launchpad.net/trove
• Community help
– #openstack-trove on Freenode
• Events
– http://www.tesora.com/troveday/
55
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle OpenStack Resources
• Product web page (data sheet, white paper, videos, FAQ)
– http://www.oracle.com/us/technologies/linux/openstack/overview/index.html
• Documentation: Installation and User Guide, HOL
– http://www.oracle.com/technetwork/server-storage/openstack/linux/documentation/index.html
• Downloads – http://www.oracle.com/technetwork/server-
storage/openstack/linux/downloads/index.html
56
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Resources
• Manual
– https://dev.mysql.com/doc/refman/5.7/en/
• Additional info
– https://www.mysql.com/products/enterprise/
• Community forums – http://forums.mysql.com/
• Server Team Blog
– http://mysqlserverteam.com
• Replication and HA Team Blog
– http://mysqlhighavailability.com
57
top related