mysql tech tour keynote, updated with 5.7.7 rc
TRANSCRIPT
The State of the Dolphin
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Nearly 5 Years of Oracle Stewardship More Investment, More Innovation
Jan 2010 Sept 2014
3
2x Engineering Staff 3x QA Staff 2x Support Staff
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Monitor 2.2
MySQL Cluster 7.1
MySQL Cluster Manager 1.0
MySQL Workbench 5.2
MySQL Database 5.5
MySQL Enterprise Backup 3.5
MySQL Enterprise Monitor 2.3
MySQL Cluster Manager 1.1
MySQL Enterprise Backup 3.7
All GA!
Oracle Products Certifications
MySQL Windows Installer
MySQL Enterprise Security
MySQL Enterprise Scalability
MySQL Enterprise Audit
MySQL Cluster 7.2
MySQL Cluster Manager 1.3
MySQL Utilities
MySQL Workbench 6.0
All GA!
MySQL Windows Tools
MySQL Database 5.6
MySQL Cluster 7.3
MySQL Enterprise Monitor 3.0
MySQL Workbench 6.1
MySQL Enterprise Backup 3.11
MySQL Fabric
MySQL Workbench 6.2
MySQL Database 5.7 DMRs*
MySQL Cluster 7.4 DMR
*Development Milestone Release
All GA!
Available Now!
Driving MySQL Innovation: 2010 - 2014
Partial List of Releases Delivered
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Reaching Out to the MySQL Community
• Sponsored 50+ events worldwide in the past year
• Delivered 60+ talks at conferences
• Dozens of Tech Tours, Developer Days, Workshops and User Groups
• Blogs & social media
• Proactive outreach to users
• Communication & collaboration with Linux distributions and popular open source projects
• MySQL Central @ OpenWorld
Sharing Information, Soliciting Feedback, Providing Help
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Complete Solutions
• Best of breed components at every level of the stack
• Complete: Meets most customer requirements
That’s why MySQL Matters to Oracle and its Customers
On Premises and in the Cloud
6
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Best Choice for Next Generation Web & Cloud Applications
World’s Most Popular Open Source Database
Leading Database for Web Applications
#1 Database in the Cloud Integrated with Hadoop in
Big Data Platforms
Strong MySQL Momentum
7
MySQL Today & Tomorrow
Mario Beck – [email protected] MySQL Sales Consulting Manager EMEA http://mablomy.blogspot.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
9
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
What’s New
• MySQL 5.7.7 Release Candidate 1
• MySQL Fabric 1.5 GA
• MySQL Workbench 6.2 GA
• MySQL Cluster 7.4 GA
• MySQL Enterprise Edition
– MySQL Enterprise Firewall
– Oracle Enterprise Manager for MySQL
• Early Access Features via labs.mysql.com
10
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Simple and convenient way to install & update MySQL products
• Supports the following distributions
– Oracle, Red Hat, CentOS
– SUSE
– Fedora
– Ubuntu, Debian
• Coming Soon
– Preconfigured Containers (Docker)
– Improved support for popular DevOps deployment tools
• The latest releases of
– MySQL Database
– MySQL Workbench
– MySQL Connector/ODBC
– MySQL Connector/Python
– MySQL Connector/NET
– MySQL Utilities
MySQL Community Repositories: Yum, APT, NuGET
11
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
- MySQL is Open Source (GPL 2) (-> http://dev.mysql.com/downloads/mysql)
Git for MySQL Engineering, GitHub for MySQL Community
- MySQL Lab releases MySQL (-> http://labs.mysql.com/)
Only for feature testing, not fit for production
- MySQL Development Milestone Releases (-> http://dev.mysql.com/downloads/mysql)
Only for testing, not fit for production
- MySQL Community Edition (-> http://www.mysql.com/downloads/mysql/)
Optional Oracle Support offering available
- MySQL Commercial Editions (-> http://www.mysql.com/downloads/ )
Oracle Support and special tools for easy DB operation included
MySQL – Transparent Development MySQL Database and MySQL Cluster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
- MySQL is Open Source (GPL 2) (-> http://dev.mysql.com/downloads/mysql)
Git for MySQL Engineering, GitHub for MySQL Community
- MySQL Lab releases MySQL (-> http://labs.mysql.com/)
Only for feature testing, not fit for production
- MySQL Development Milestone Releases (-> http://dev.mysql.com/downloads/mysql)
Only for testing, not fit for production
- MySQL Community Edition (-> http://www.mysql.com/downloads/mysql/)
Optional Oracle Support offering available
- MySQL Commercial Editions (-> http://www.mysql.com/downloads/ )
Oracle Support and special tools for easy DB operation included
MySQL – Transparent Development MySQL Database and MySQL Cluster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Source Code Available on GitHub • Git for MySQL Engineering
– Fast, flexible and great for a distributed team
– Great tooling
– Large and vibrant community
• GitHub for MySQL Community
– Easy and fast code availability to the community and to downstream projects
– Repo available today at https://github.com/mysql
– More info at http://mysqlrelease.com
14
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
- MySQL is Open Source (GPL 2) (-> http://dev.mysql.com/downloads/mysql)
Git for MySQL Engineering, GitHub for MySQL Community
- MySQL Lab releases MySQL (-> http://labs.mysql.com/)
Only for feature testing, not fit for production
- MySQL Development Milestone Releases (-> http://dev.mysql.com/downloads/mysql)
Only for testing, not fit for production
- MySQL Community Edition (-> http://www.mysql.com/downloads/mysql/)
Optional Oracle Support offering available
- MySQL Commercial Editions (-> http://www.mysql.com/downloads/ )
Oracle Support and special tools for easy DB operation included
MySQL – Transparent Development MySQL Database and MySQL Cluster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Replacing the FRMs
• Single repository of metadata for Storage Engine and MySQL server
– Leveraging InnoDB Strengths
– Reliable crash-safe tables
• INFORMATION_SCHEMA tables can be simple VIEWs – Queries can be optimized
– Improved performance
• Extensible
– Designed to provide backward-compatibility
– Designed with plugins in mind
New Data Dictionary
labs.mysql.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Shared-nothing virtually synchronous database system
• Multi-master update anywhere
– Conflict detection and resolution (transaction rollback)
– Optimistic State Machine Replication
• Automatic group membership management and failure detection
– No need for server fail-over
– Elastic scale out/in
– No single point of failure
– Automatic reconfiguration
• Well integrated
– InnoDB
– GTID-based replication
– PERFORMANCE_SCHEMA
MySQL Group Replication
Application
MySQL Masters Replication
Plugin
API
MySQL Server
Group Comms (Corosync)
17
labs.mysql.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Improving Cloud Operations
• Provisions machines & MySQL
– Initially using OpenStack Nova
– Other cloud frameworks on the way (OpenStack Trove, AWS,…)
• Server setup
– Clones slave
– Sets up replication
– Performs custom operations
18
MySQL Fabric: OpenStack Server Provisioning
labs.mysql.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
HTTP Plugin for MySQL
• Server Plugin adds HTTP(S) endpoints to MySQL
• Results are serialized to JSON format encoded as UTF8
• HTTP web service authenticates user/host using MySQL authentication
• Provides 3 choices of User Endpoint Types
– SQL
– CRUD - Key-Value
– JSON - Document
19
labs.mysql.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
JSON support
• New data type: JSON
• Server side JSON functions ( jsn_valid(...), jsn_extract(...), ...)
• Create functional indexes on JSON elements
– CREATE TABLE employees(id BIGINT not null primary key auto_increment, info JSON);
– ALTER TABLE employees ADD COLUMN name VARCHAR(100) GENERATED ALWAYS AS (jsn_extract(info, '$.name')) VIRTUAL;
– ALTER TABLE employees ADD INDEX (name);
20
labs.mysql.com
labs.mysql.com
RC 1
labs.mysql.com
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
- MySQL is Open Source (GPL 2) (-> http://dev.mysql.com/downloads/mysql)
Git for MySQL Engineering, GitHub for MySQL Community
- MySQL Lab releases MySQL (-> http://labs.mysql.com/)
Only for feature testing, not fit for production
- MySQL Development Milestone Releases (-> http://dev.mysql.com/downloads/mysql)
Only for testing, not fit for production
- MySQL Community Edition (-> http://www.mysql.com/downloads/mysql/)
Optional Oracle Support offering available
- MySQL Commercial Editions (-> http://www.mysql.com/downloads/ )
Oracle Support and special tools for easy DB operation included
MySQL – Transparent Development MySQL Database and MySQL Cluster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Improvements
• InnoDB for better transactional throughput, availability, IO
• Replication for better scalability and availability
• Fabric for high availability and sharding
• Performance Schema for new and improved performance metrics
• Optimizer for better EXPLAINing, parsing, query performance
• GIS with native InnoDB spatial indexes and Boost.Geometry integration
Available Now! dev.mysql.com/downloads/mysql/
22
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Optimizer – New Cost Model Improving Query Performance
• New cost model enables storage engines to provide
– Accurate and dynamic cost estimates
– Key lookups, table scans, range scans, index scans, etc…
• Configurable costs for additional factors – Speed of disk I/O operations
– Speed of memory operations
• Improved records per key estimates
• Cost values included in JSON Explain output
23
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Optimizer - Query Rewrite Plugin
• Query rewrite (pre and post parse)
• Post-parse query plugin
– Rewrite problematic queries without application changes
– Add hints
– Modify join order
– Many more …
• Improve problematic queries from ORMs, third party apps, etc.
24
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Memory Instrumentation
• Aggregates statistics by
– Type of memory used (caches, internal buffers, …)
– Thread/account/user/host indirectly performing the memory operation
• Attributes include
– Memory used (bytes)
– Operation counts
– High/Low Water Marks
Statement Instrumentation
• Stored Procedures
• Stored Functions
• Prepared Statements
• Transactions
Additional Information
• Replication slave status
• MDL lock instrumentation
• User variables per thread
• Server stage tracking
• Track long running SQL
• All while reducing total footprint and overhead
MySQL 5.7: Performance Schema Improvements
25
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7 : Syslog Support for Linux/Unix platforms
• Native support for syslog
• Simple option to (re)direct log output to native syslog facility
• Start-up server configuration option
• Dynamically in the running server
– System variable log_syslog (ON/OFF, defaults to OFF).
26
Thank you, Simon Mudd at booking.com
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Consolidate updates from multiple Masters into one Slave
– Consolidated view of all shards
– More flexible topologies
– Centralized point for backups
• Compatible with Semi-Sync Replication & enhanced MTS
• Master-specific slave filters
Multi-Source Replication
Binlog
Master 1
Binlog
Master 2
…
…
Binlog
Master N
IO 1
Relay 1
Coordinator
W1 W2 … WX
IO 2
Relay 2
Coordinator
W1 W2 … WX
…
…
Coordinator
W1 W2 … WX
IO N
Relay N
Coordinator
W1 W2 … WX
Slave
27
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Replication Improvements • Improved Master throughput
– Better synchronization between replication-user sessions; Ack receiver thread, group commit tuning…
• Multi-Threaded Slaves:
– Performance - Slave applies transactions in parallel even within same database; group commit tuning
– Option to preserve Commit order
– Automatic slave transaction retries
• Use GTIDs without binary logging on slave
• Lossless Replication through enhanced Semi-sync
• Configurable number of Semi-sync slave Acks
• Performance Schema tables for monitoring slave
• Online Operations: Dynamic Replication Filters, switch master
28
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Replaced custom code
– For spatial calculations
– For spatial analysis
• Provides OGC compliance – With improved performance
• Boost.Geometry contains
– Field and domain experts
– Bustling and robust community
• R-Tree Indexes on geo data
MySQL 5.7: GIS - Integrating Boost.Geometry
29
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Security - Encryption, Passwords, Installation
• AES 256 Encryption
– Default in MySQL 5.7
• Password rotation policies
– Can be set globally, and at the user level
• Deployment: enable secure unattended install by default – Random password set on install
– Remove anonymous accounts
– Deployment without test account, schema, demo files
30
RC 1
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
0
100.000
200.000
300.000
400.000
500.000
600.000
700.000
8 16 32 64 128 256 512 1.024
Qu
eri
es
pe
r Se
con
d
Connections
MySQL 5.7: Sysbench Read Only (Point Select)
MySQL 5.7
MySQL 5.6
MySQL 5.5
MySQL 5.7: Sysbench Benchmark
Intel(R) Xeon(R) CPU E7-4860 x86_64 4 sockets x 10 cores-HT (80 CPU threads) 2.3 GHz, 512 GB RAM Oracle Linux 6.5
2x Faster than MySQL 5.6 3x Faster than MySQL 5.5
645,000 QPS
31
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: InnoDB, NoSQL With Memcached 6x Faster than MySQL 5.6 Thank you, Facebook
0
200.000
400.000
600.000
800.000
1.000.000
1.200.000
8 16 32 64 128 256 512 1.024
Qu
eri
es
pe
r Se
con
d
Connections
MySQL 5.7 vs 5.6 - InnoDB & Memcached
MySQL 5.7
MySQL 5.6
1 Million QPS
32
Intel(R) Xeon(R) CPU E7-4860 x86_64 4 sockets x 10 cores-HT (80 CPU threads) 2.3 GHz, 512 GB RAM Oracle Linux 6.5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL 5.7: Connections per Second 1.7x Faster than MySQL 5.6 2.5x Faster than MySQL 5.5
67,000 Connections/Sec
0
10.000
20.000
30.000
40.000
50.000
60.000
70.000
80.000
MySQL 5.5 MySQL 5.6 MySQL 5.7
Co
nn
ect
ion
s/Se
con
d
Connections Per Second
MySQL 5.5
MySQL 5.6
MySQL 5.7
33
Intel(R) Xeon(R) CPU E7-4860 x86_64 4 sockets x 10 cores-HT (80 CPU threads) 2.3 GHz, 512 GB RAM Oracle Linux 6.5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
- MySQL is Open Source (GPL 2) (-> http://dev.mysql.com/downloads/mysql)
Git for MySQL Engineering, GitHub for MySQL Community
- MySQL Lab releases MySQL (-> http://labs.mysql.com/)
Only for feature testing, not fit for production
- MySQL Development Milestone Releases (-> http://dev.mysql.com/downloads/mysql)
Only for testing, not fit for production
- MySQL Community Edition (-> http://www.mysql.com/downloads/mysql/)
Optional Oracle Support offering available
- MySQL Commercial Editions (-> http://www.mysql.com/downloads/ )
Oracle Support and special tools for easy DB operation included
MySQL – Transparent Development MySQL Database and MySQL Cluster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Management Tools Advanced Features Support
• Scalability • High Availability • Security • Audit
•Monitoring • Backup • Development • Administration •Migration
• Technical Support • Consultative Support •Oracle Certifications
MySQL Enterprise Edition
35
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Firewall
• Real Time Protection
– Queries analyzed and matched against White List
• Blocks SQL Injection Attacks
– Positive Security Model
• Block Suspicious Traffic – Out of Policy Transactions detected & blocked
• Learns White List
– Automated creation of approved list of SQL command patterns on a per user basis
• Transparent
– No changes to application required
36
MySQL Enterprise Firewall monitoring
GA
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Firewall
• SQL Injection Protection with Positive Security Model
37
• Out of policy database transactions detected and blocked
• Logging & Analysis
Select *.* from employee where id=22
Select *.* from employee where id=22 or 1=1
Block & Log ✖
Allow & Log ✔
White List Applications
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Encryption Standards Based Cryptography
• MySQL encryption libraries
– Symmetric encryption AES256
– Public-key / asymmetric cryptography
• Key management – Generate public and private keys
– Key exchange methods: RSA, DSA, DH
• Sign and verify data
– Cryptographic hashing for digital signing, verification, & validation
38
GA
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Single Dashboard to manage Oracle stack for Web & Cloud
• An estimated 70% of Oracle customers also use MySQL
– For Web, custom, departmental and embedded applications
– #1 Requested integration
40
Oracle Enterprise Manager for MySQL GA
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Enterprise Manager for MySQL
41
Performance Security
Availability
• Availability monitoring
• Performance monitoring
• Configuration monitoring
• All available metrics collected
– Allowing for custom threshold based incident reports
• MySQL auto-detection
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Management Tools Advanced Features Support
• Scalability • High Availability • Security • Audit
•Monitoring • Backup • Development • Administration •Migration
• Technical Support • Consultative Support •Oracle Certifications
MySQL Enterprise Edition
42
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Thank You!
Mario Beck Mobile: +49 172 852 04 29 [email protected]