driving innovation with mysql and dockermysql server 8.0: agenda top five 8.0 features for...

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |

Driving Innovation with MySQL and Docker

Kathy Forte MySQL Curriculum Developer Microservices World 2019 (With Insights from MySQL Development and Product Management Teams)

Copyright © 2019, Oracle and/or its affiliates. All rights reserved.

MySQL 8.0


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2


Feature Request from DBAs Feature Request

from Developers

Many Feature Requests in 8.0 Came From You


MySQL Server 8.0: The Oracle MySQL Team Writes the Code

Drawing from https://www.dreamstime.com/royalty-free-stock-image-source-code-puzzle-shows-software-program-programming-showing-image34210956


Scalable & Secure Improved query/index maintenance tools, security tools, and performance improvements

Data Driven Optimizing services with real time data analysis

MySQL 8.0 - Enables Modern Web Applications


24x7 at Scale

Developer First Hybrid data model and data access APIs for flexibility for developers

Mobile Friendly Ready for location based services. Handling Emoji and Unicode characters


https://modern-sql.com/blog/2019-04/mysql-8.0

Quote from External, Independent Source

https://twitter.com/MarkusWinand

“Let me therefore boldly claim: MySQL 8.0 is one small step for a database, one giant leap for SQL” - Markus Winand, Modern SQL

https://dev.mysql.com/doc/refman/8.0/en/upgrading.html/









https://twitter.com/MarkusWinand



MySQL Server 8.0: Agenda

Top Five 8.0 Features for Developers

Top Five 8.0 Features for DBAs

Containers and MySQL

2

3

1



MySQL – Next Version – 8.0: Top Features for Developers

1) MySQL Document Store

2) Window Functions or Analytical Functions

3) Common Table Expressions (CTEs)

4) InnoDB NO WAIT & SKIP LOCKED

5) utf8mb4 Character Set

12

/


MySQL 8.0 – Hybrid Model

MySQL Document Store – NoSQL within MySQL product

12

Developer First Hybrid data model and data access APIs for flexibility for developers


https://www.youtube.com/watch?v=E8zwnXjIjPM

https://www.youtube.com/watch?v=E8zwnXjIjPM


1. MySQL Document Store

12

Business Benefit: Both worlds in one database product:

Saves time - don’t have to learn yet another database product

Reduces complexity of dealing with multiple vendors


Relational vs. Document Store (Non-Relational)

13

https://www.upwork.com/hiring/data/sql-vs-nosql-databases-whats-the-difference/


MySQL as a Document Store

• All the existing features of MySQL – Replication

– ACID transaction

– Documents have data integrity

– All of our tools

– Collection joins; analytics on collections

• With the addition of schemaless – Documents using JSON

– Easy to program CRUD APIs

14

http://datastoragecharlotte.com/services/document-storage/


CRUD Operations NoSQL/Document Javascript Java

C# NodeJS


Document Store

How to use Document Store within MySQL: https://dev.mysql.com/doc/refman/8.0/en/document-store.html

• How to combine SQL and NoSQL (video): https://www.youtube.com/watch?v=yQvdJ2mVHjg&index=2&list=PLWx5a9Tn2EvGe-LGUpXYkv8-5LyKGh_DF

• Blogs - MySQL Document Store:

https://mysqlserverteam.com/category/docstore/

17

https://dev.mysql.com/doc/refman/8.0/en/document-store.html





https://www.youtube.com/watch?v=yQvdJ2mVHjg&index=2&list=PLWx5a9Tn2EvGe-LGUpXYkv8-5LyKGh_DF











MySQL 8.0 - Data-Driven Models for Developers

More Top Features for SQL Developers

12

/

Data Driven Optimizing services with real time data analysis and locking control


MySQL 8.0 – Data-Driven Models

12

Business Benefit: Real-time data analysis and more:

Less complex programming with Window Functions, Common Table Expressions (CTEs), ‘NO WAIT’ & ‘SKIP LOCKED’

Saves time - don’t have to switch to analytics database


2. MySQL Window Functions

20

Improving Support for Analytics

https://www.google.com/search?biw=1646&bih=907&tbm=isch&sa=1&q=window++cat&oq=window++cat&gs_l=psy-ab.3..0l6j0i67k1j0l3.20057.20669.0.21178.4.4.0.0.0.0.217.601.0j3j1.4.0....0...1.1.64.psy-ab..0.2.211...0i7i30k1j0i8i7i30k1.0.lMDZgoQOoG0#imgrc=w_TkHuUSM07wtM:

Feature Request from Developers


Window Functions

21

Improving Support for Analytics

Window functions gather results into a set of rows, returning a single aggregated value for each of the rows in the result set

https://www.google.com/search?biw=1646&bih=907&tbm=isch&sa=1&q=window++cat&oq=window++cat&gs_l=psy-ab.3..0l6j0i67k1j0l3.20057.20669.0.21178.4.4.0.0.0.0.217.601.0j3j1.4.0....0...1.1.64.psy-ab..0.2.211...0i7i30k1j0i8i7i30k1.0.lMDZgoQOoG0#imgrc=w_TkHuUSM07wtM:


Window Functions

21

SELECT *

FROM player;

The top ten batters in the National League - 2017

Justin Turner Dodgers 348

Charlie Blackman Rockies 333

Daniel Murphy Nationals 332

Bryce Harper Nationals 327

Paul Goldschmidt Diamondbacks 322

Buster Posey Giants 321

DJ LeMahieu Rockies 316

Nolan Arenado Rockies 315

Joey Votto Reds 315

Ryan Zimmerman Nationals 310

Consider:


Window Functions

23

Here’s an aggregate function: give me the top batting average from each team of top ten batters for each NL team:

SELECT team, MAX(baverage)

FROM player

GROUP BY team;

Diamondbacks 322

Nationals 332

Dodgers 348

Giants 321

Reds 315

Rockies 333


Window Functions

24

SELECT team, name, baverage,

MAX(baverage)

OVER (PARTITION BY team)

FROM player;

24

Team Name baverage MAX (baverage)

Diamondbacks Paul Goldschmidt 322 322

Dodgers Justin Turner 348 348

Giants Buster Posey 321 321

Nationals Ryan Zimmerman 310 332

Nationals

Daniel Murphy 332 332

Nationals

Bryce Harper 327 332

Reds Joey Votto 315 315

Rockies Charlie Blackmon 333 333

Rockies DJ LeMahieu 316 333

Rockies Nolan Arenado 315 333


Window Functions

25

SELECT team, name, baverage,

RANK() OVER (PARTITION BY team

ORDER BY baverage DESC)

FROM player

ORDER BY team, baverage DESC;

25

Diamondbacks Paul Goldschmidt 322 1

Dodgers Justin Turner 348 1

Giants Buster Posey 321 1

Nationals

Daniel Murphy

332 1

Nationals

Bryce Harper

327 2

Nationals Ryan Zimmerman 310 3

Reds Joey Votto

315 1

Rockies Charlie Blackmon 333 1

Rockies DJ LeMahieu 316 2

Rockies Nolan Arenado 315 3


More on Window Functions

http://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/

https://dev.mysql.com/doc/refman/8.0/en/window-functions.html



















3. MySQL Common Table Expressions (CTEs)

27


https://forums.anandtech.com/threads/could-you-live-a-recursive-life.2393849/


Common Table Expressions (CTEs)

• “With queries” – Statement scoped views

– Valid only in the query they are being used

– Benefit: Simplifies writing complex SQL: WITH t1 AS (SELECT * FROM tblA WHERE a=‘b’) SELECT * FROM t1;

28

https://forums.anandtech.com/threads/could-you-live-a-recursive-life.2393849/


Recursive CTE

a

1

2

3

4

5

6

7

8

9

10

WITH RECURSIVE qn AS

(

SELECT 1 AS a

UNION ALL

SELECT 1+a FROM qn WHERE a<10

)

SELECT * FROM qn; https://forums.anandtech.com/threads/could-you-live-a-recursive-life.2393849/


Family Tree


Benefit: Iteration Consider: A table with one parent, one child, family tree

Recursive CTE

parent child

Charles Jeffrey

Jeffrey Kathy

Kathy Samuel


Benefit: Iteration How many levels in this family tree?

Recursive CTE

WITH RECURSIVE descendants AS ( SELECT parent, child AS descendant, 1 as level FROM family UNION ALL SELECT d.parent, f.child, d.level + 1 FROM descendants as d JOIN family f ON d.descendant = f.parent ) SELECT * FROM descendants ORDER BY parent, level, descendant ;

parent descendant level

Charles Jeffrey 1

Charles Kathy 2

Charles Samuel 3

descendants

…..


More on CTEs First blog in a series:

http://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/

Performance and CTEs:

https://mysqlserverteam.com/mysql-8-0-improved-performance-with-cte/





































4. MySQL Hot rows – Handle with SKIP LOCKS and NOWAIT Constructs

• https://workwiththebest.intraway.com/blog-post/the-use-of-oracle-feature-select-for-update-to-prevent-race-conditions-on-our-exposed-services/



Locking – Control Over Hot Rows

https://workwiththebest.intraway.com/blog-post/the-use-of-oracle-feature-select-for-update-to-prevent-race-conditions-on-our-exposed-services/


Locking

36


http://mysqlserverteam.com/mysql-8-0-1-using-skip-locked-and-nowait-to-handle-hot-rows/ https://dev.mysql.com/doc/refman/8.0/en/data-locks-table.html

http://mysqlserverteam.com/mysql-8-0-1-using-skip-locked-and-nowait-to-handle-hot-rows/


























https://dev.mysql.com/doc/refman/8.0/en/data-locks-table.html






• MySQL 8.0 defaults to utf8mb4

• Latest Unicode 9.0 Support

– Emoji characters

• Support for 21 languages!

5. MySQL 8.0: utf8mb4 Encoding

https://www.google.com/search?tbm=isch&q=emojis+happy&chips=q:emoji+happy,g_3:happiness&sa=X&ved=0ahUKEwjo9c6q9ZnbAhWIAnwKHeqpBwwQ4lYIKigA&biw=1860&bih=846&dpr=0.8#imgrc=OFM

SFSANDEYHDM:






2

3

1



MySQL – Next Version – 8.0: Top Features for DBAs

6) Invisible Indexes

7) Check Constraints

8) Roles

9) Transactional Data Dictionary

10) Performance

12


MySQL 8.0 – Top Features for DBAs

12

/

Scalable & Secure Improved query/index maintenance tools, security tools, and performance improvements

24x7 at Scale


MySQL 8.0 – For DBAs

12

Business Benefits of New Features:

More control of index placement with ‘invisible’ indexes

Improves data integrity with check constraints

Reduces complexity of setting up security with roles

Better performance and reliability with data dictionary


6. MySQL Invisible Indexes – Toggling Indexes ‘Off’ and ‘On’

42

https://www.google.com/search?q=invisible+man+old+movie&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjwhJaj0vnaAhUCwFQKHZTiD7sQ_AUICygC&biw=1422&bih=1002#imgrc=l5aW4QqAIIEqdM:

Feature Request from DBAs


Invisible Indexes

• Indexes are “hidden” from the MySQL Optimizer

– Index pages are updated but not used in queries by optimizer

43

http://viralpatel.net/blogs/invisible-indees-in-oracle-11g/



Invisible Indexes – Toggling Indexes ‘Off’ and ‘On’

• Two use cases:

–Soft Delete (Recycle Bin)

–Staged Rollout

44



Invisible Indexes - Soft Delete

• “I don’t think this index is used any more": ALTER TABLE Country ALTER INDEX c INVISIBLE;

• “Queries not slower. Okay, it is now safe to drop the index”: ALTER TABLE Country DROP INDEX c;



Invisible Indexes - Staged Rollout

• Adding any new index can change existing execution plans

• All change introduces risk of regression ALTER TABLE Country ADD INDEX c (Continent) INVISIBLE;

• “I want to test just in my session”

SET optimizer_switch=‘use_invisible_indexes=on’;

• “Okay, it works after my experiments, turn it on” ALTER TABLE Country ALTER INDEX c VISIBLE;

46

https://www.google.com/search?q=invisible+man+old+movie&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjwhJaj0vnaAhUCwFQKHZTiD7sQ_AUICygC&biw=1422&bih=1002#imgrc=l5aW4QqAIIEqdM


http://mysqlserverteam.com/mysql-8-0-invisible-indexes/

More on Invisible Indexes











7. MySQL Check Constraints

• Remove checks from your programs and place in the database where they belong

• But once defined, many of those rules can be enforced by the DBMS, thereby improving data quality … and your business processes.

Semantic integrity of the data



• INSERT and UPDATE processing) will cause the expression to be evaluated

- If the modification conforms to the Boolean expression, the modification is permitted to continue

- not, the statement will fail with a constraint violation.

Semantic integrity of the data



CREATE TABLE products

(

product_no INTEGER,

name TEXT,

price NUMERIC CHECK (price > 0),

cost_to_produce NUMERIC CHECK (cost_to_produce > 0),

CHECK (price > cost_to_produce)

);

INSERT INTO products VALUES (1, 'gum', 1, 1);

ERROR 3819 (HY000): Check constraint 'products_chk_3‘

is violated.

Here’s my latest blog on check constraints: ‘MYSQL CHECK CONSTRAINTS TO THE RESCUE’ (https://mysqlsolutionsblog.wordpress.com/2019/09/24/mysql-check-constraints-to-the-rescue/)

https://mysqlsolutionsblog.wordpress.com/2019/09/24/mysql-check-constraints-to-the-rescue/
















8. MySQL Roles Improving MySQL Access Controls

51

Feature Request from DBAs

https://cougardaily.org/7241/student-life/advanced-acting-and-advanced-drawing-draw-together/


MySQL Roles

Improving MySQL Access Controls

52



MySQL Roles

Improving MySQL Access Controls

• Easier to manage user and applications rights

• As standards compliant as practically possible

• Multiple default roles

https://dev.mysql.com/doc/refman/8.0/en/roles.html

53


https://dev.mysql.com/doc/refman/8.0/en/roles.html


9. MySQL Transactional Data Dictionary

• Increased reliability and performance

• Using InnoDB tables internally for data dictionary – No FRM files

–No DB.OPT files

– No TRG files

– No TRN files

– No PAR files

54

Data Dictionary

Increased reliability and consistency with transactional meta data repository


Transactional Data Dictionary

• Better cross-platform experience

–No dependencies on filesystem semantics

• Atomic DDL

– Robust replication (Group Replication)

• Flexible metadata API

– Easier path to adding new features

55

Additional Benefits

Data Dictionary

Increased reliability and consistency with transactional meta data repository


Atomic Statements

• Applies to statements performing multiple logical operations:

–DROP TABLE t1, t2;

• Uses a custom Meta Data Lock (MDL) for Data Definition Language (DDL)

56


Instant Add Column

• Store information in system catalogues and only changes your table when you add new rows:

–ALTER TABLE t1 ADD COLUMN c3 INT DEFAULT 1, ALGORITHM = INSTANT;

• Cannot ‘ALTER DROP COLUMN’ (coming in later releases)

57


8.0 Data Dictionary

http://mysqlserverteam.com/mysql-8-0-data-dictionary-background-and-motivation/

58

















10. MySQL Performance

MySQL 8.0 is 2x faster than MySQL 5.7 and sets new benchmarks records


MySQL 8.0: SysBench IO Bound Read Only (Point Selects) 2x Faster than MySQL 5.7

60

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1 2 4 8 16 32 64 128 256 512

Qu

eri

es

pe

r Se

con

d

Users

MySQL 8.0

MySQL 5.7

MySQL 5.6

OS : Oracle Linux 7.4 CPU : 48cores-HT Intel Skylake 2.7Ghz (2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU) RAM: 256GB Storage : x2 Intel Optane flash devices (Intel (R) Optane (TM) SSD P4800X Series)


0

50,000

100,000

150,000

200,000

250,000

300,000

1 2 4 8 16 32 64 128 256 512 1,024

Qu

eri

es

pe

r Se

con

d

Users

MySQL 8.0

MySQL 5.7

MySQL 5.6

MySQL 8.0: SysBench Read Write (update nokey) ~2x Faster than MySQL 5.7

61

OS : Oracle Linux 7.4 CPU : 48cores-HT Intel Skylake 2.7Ghz (2CPU sockets, Intel(R) Xeon(R) Platinum 8168 CPU) RAM: 256GB Storage : x2 Intel Optane flash devices (Intel (R) Optane (TM) SSD P4800X Series)


More Concurrency Improves Performance

Old Design New Design


Performance Improvements

• For more information on the MySQL 8.0 benchmarks:

http://www.mysql.com/why-mysql/benchmarks/

• The SysBench benchmark tool is freely available for specific benchmarks and can be downloaded here: http://dev.mysql.com/downloads/benchmarks.html

• And this blog: http://dimitrik.free.fr/blog/






http://dev.mysql.com/downloads/benchmarks.html

http://dev.mysql.com/downloads/benchmarks.html

http://dimitrik.free.fr/blog/


Benefit of Indexes: Performance Comparison

0. 10. 20. 30. 40.

Query Time

MySQL 8.0MySQL 5.7

Over 30x faster!

1000 active sessions

Time in Seconds (Lower is better)

SELECT TABLE_SCHEMA,

TABLE_NAME, TABLE_TYPE,

ENGINE, ROW_FORMAT

FROM information_schema.tables

WHERE TABLE_SCHEMA LIKE 'db%';






2

3

1



VMS versus Docker


Why Docker

“You create an illusion for your application that it is getting its very own OS instance, although there may be other containers running on same system. Docker is what enables us to run, create and manage containers on a single operating system.”

https://containerjournal.com/2019/01/14/kubernetes-vs-docker-a-primer/












Why Docker with MySQL

• Advantages of MySQL with Docker - Fastest way to test 8.0! - Portable - Easy to use - Lightweight (less than 1GB)


Why Docker with MySQL

Uber is using MySQL with Docker: https://eng.uber.com/dockerizing-mysql/

https://eng.uber.com/dockerizing-mysql/




Try Docker With MySQL

• First, install Docker: https://docs.docker.com/ or go to https://labs.play-with-docker.com/ for online Alpine sandbox

• Then, pull 8.0 and play with it: docker run --name mysql8 -e MYSQL_ROOT_PASSWORD=Newpasswd# -d

mysql/mysql-server

docker exec -it mysql8 mysql -uroot –p

Enter password:

https://docs.docker.com/

https://labs.play-with-docker.com/








For More Information on Docker and MySQL

https://mysqlsolutionsblog.wordpress

.com/2018/09/25/driving-innovation-

with-mysql-8-0-and-docker/

My blog on setting up 8.0 for testing:

https://mysqlsolutionsblog.wordpress.com/2018/09/25/driving-innovation-with-mysql-8-0-and-docker/


















Something Extra - MySQL Shell – Dual SQL/CRUD Tool for Prototyping


Something Extra - Try MySQL 8.0 Document Store with Docker and MySQL Shell

• First, install Docker: https://docs.docker.com/ or go to https://labs.play-with-docker.com/ for online Alpine Linux sandbox

• Then, pull 8.0 and connect to MySQL Shell: $ docker run --name mysql8 -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql/mysql-

server

/* have to point to the directory of mysqlx.sock */

$ docker exec -it mysql8 mysqlsh --mysqlx -uroot -S/var/run/mysqld/mysqlx.sock

Confidential - Oracle Internal 73

https://docs.docker.com/







MySQL Document Store – MySQL Shell

Confidential – Oracle Internal/Restricted/Highly Restricted 74

http://datastoragecharlotte.com/services/document-storage


MySQL Document Store – MySQL Shell Move Back and Forth from CRUD to SQL

Confidential – Oracle Internal/Restricted/Highly Restricted 75

http://datastoragecharlotte.com/services/document-storage


More Information on Docker and MySQL Shell

https://mysqlsolutionsblog.wordpress

.com/2018/09/27/running-mysql-shell-

8-0-with-docker/

My blog on setting up MySQL 8.0 with MySQL Shell for testing Document Store:

https://mysqlsolutionsblog.wordpress.com/2018/09/27/running-mysql-shell-8-0-with-docker/















MySQL Docker Enterprise Edition

$ docker login container-registry.oracle.com

Username: [email protected]

Password:

$ docker pull container-registry.oracle.com/mysql/enterprise-server:8.0.13

8.0.13: Pulling from mysql/enterprise-server

e64f6e679e1a: Pull complete

Docker Engine


Oracle MySQL Operator – Alpha Code

78

Agent

Operator

Kubernetes API

Pod (my-cluster-N)

Object storage

CRD

MySQLCluster

MySQLBackup

MySQLScheduledBackup

MySQLRestore

MstatefulSet (my-cluster)


Information on Latest Release 8.0.17

https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-17.html

Release Notes for Latest Release 8.0.17:

Blog on 8.0.17: https://mysqlserverteam.com/the-mysql-8-0-17-maintenance-

release-is-generally-available/









https://mysqlserverteam.com/the-mysql-8-0-17-maintenance-release-is-generally-available/




















https://dev.mysql.com/doc/refman/8.0/en/upgrading.html/

Upgrade to 8.0 Information:

• Strategies for upgrade, changes:

• Tool to check for incompatibilities (from 5.7 to 8.0): http://lefred.be/content/how-to-safely-upgrade-to-mysql-8-0/

https://dev.mysql.com/doc/refman/8.0/en/upgrading.html

http://lefred.be/content/how-to-safely-upgrade-to-mysql-8-0/

















• CTEs

• Window Functions

• SKIP LOCKED and NOWAIT (hot row handling)

• Document Store

• Unicode (utf8mb4 default) encoding

• Data Dictionary

• Roles

• Invisible Indexes

• Histograms

• Descending Indexes

• Redo Undo Log Encryption

• Binary and Relay Log Encryption

• Password Rotation Policy

• GIS Improvements (3-D)

• Scalability Improvements

• Atomic DDL

• Cost Model Improvements

• Replication Applier Thread Progress Reports

• GTID_PURGED always settable

• Undo tablespaces now dynamic

• Smaller Package Downloads

• JSON table function

• JSON path expressions (ranges)

• JSON performance improvements

• Expanded Query Hints

• Improved usability of cost constant configuration

• Improved Parallel Replication

• SSD by default

• Resource Groups

• Check Constraint

• Dedicated server: self-tune

• SET PERSIST 81

Top 8.0 Features

Cloud Friendly


Q & A Your Thoughts


Thank You!

[email protected]