Upload File

froscon2011: how i learned to use sql and then learned not to use it

  • Home
  • Technology
  • Froscon2011: How i learned to use sql and then learned not to use it
29
2011-07-26 OSCON 2011 1 How I learned to use SQL and then learned not to use it Henrik Ingo Froscon 2011-08-20

Upload: henrik-ingo

Post on 17-May-2015

2.083 views

Category:

Technology


6 download

Report

Tags:

DESCRIPTION

Keynote for the Open DB Camp track (developer room) at Froscon2011. The point is to compare the history of MySQL with the evolution of new NoSQL systems.

TRANSCRIPT

Page 1: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 1

How I learned to use SQLand then learned not to use it

Henrik Ingo

Froscon2011-08-20

Page 2: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 2

Henrik Ingo

open source technology and strategy specialist

active in MySQL, Drupal communities

current: Senior Performance Architect at Nokia Ovi

worked in mobile and LAMP with business management, sales, R&D

author of "Open Life: The Philosophy of Open Source"

[email protected]

Page 3: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 3

1994

Page 4: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 4

1994

USSR fell 4 years ago

Finland yet to win first gold medal in hockey

Windows 3.0 replacing MS-DOS

Windows 95 did not yet exist

I learn how to write .bat scripts from my dad

Word 6.0 replacing WordPerfect 5.1

Excel 5.0 replacing Lotus

I had not yet used Internet

Page 5: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 5

1994

C++ & Visual Basic

LANPrinter Database

Client - Server architecture

SQLStandard Interpreted

Flexible and expressive command line env

"SQL for secretaries"Good for English speakers

2000: Bad for IDE w IntelliSense

Stored procedures rule

DBA is king of business logic

Page 6: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 6

1997

Page 7: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 7

1997

Page 8: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 8

Learning SQL

NoSQL advocates say:

SQL is hard to learn

To really scale, you must de-normalize

Most people don't get normalization right

Impedance mismatch between oo and sql

I just need a simple key-value store

Henrik says:

MS Access easy to learn

Darn, I always got normalization

I know, teaching n:n relations was always fun

INSERT INTO t ... serialize($obj)

SELECT value FROM tWHERE key=?;

Page 9: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 9

1997

WorldWideWebFree Software

1998: Open Source

LAMP stackWhere P = Perl, runs +90% of web "apps"

Oracle doesn't support Linux

PostgreSQL doesn't support Windows

MiniSQL has poor performance

Wish I had a simple data store for my web apps that was really fast and scalable and Free Software...

Page 10: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 10

1997

Page 11: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 11

1997

ERROR: Could not connect to MySQL: ERROR 1040: Too many connections

"Slashdotted"Web jargon ca 1997."Your site is down becauseit isn't web scale."

Page 12: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 12

History of MySQL

1985... ISAM key-value store - Trivia: Access is also ISAM db

SQL support added (HTML)

Data Warehousing: bulk loads, reporting. - Flexible attitude to errors during insert

ODBC (Excel) - BI tool #1

1995.. Released under FOSS-like license

Page 13: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 13

1997

"Read-mostly" - No MVCC - Not transactional

Not crash safe

Scales well for 2-3 CPUs - This will work well for the next 10 years

Not ANSI SQL

Page 14: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 14

1997

This site is currently offline for nightly backup. Please come back after 15 minutes.

(No online backups)

Page 15: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 15

1997-2007

InnoDB and BDB - Transactions!

SAP partnership - 4 CPU servers - Support SQL-92 "enterprise" features - Oracle acquires InnoDB and BDB

Replication - Statement based - Asynchronous

Online backup tool

Page 16: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 16

2007

We want a database that - does not conform to SQL standard - has no transactions - is not crash safe - impossible to do consistent backups

- but scales to more than 4 cores!

Users are not happy!

Page 17: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 17

You see where I'm going, don't you...

Page 18: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 18

Sorry, that was a cheap shot...

...I just couldn't resist :-)

Page 19: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 19

It really started with...

Page 20: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 20

Actual questions from Memcache users, ca 2007:

How can I dump all records from a memcached instance?

How can I make memcached highly available?

Backups?

Page 21: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 21

Answers from Memcache developers

It's a cache. Please use ittogether with a real database!

Why would you ever want to do that?

No no no, you're doing it wrong!

Page 22: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 22

So clearly there was a market demand...

Page 23: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 23

Things NoSQL guys do really well

No SQL (parsing)

Schemaless = Win! for agile development

HA with quorum consistency: R + W > N

Transparent sharding

Graph databases

N:N relationships, what's the big deal?

Actually makes sense to give up on SQL!

Map Reduce

Bypass ETL, get clean data

Implemented in Java or Python (or Erlang)

Page 24: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 24

Best of both worlds

NoSQLNo SQL.

Simple key-value store.

...and secondary indexes

Quorum consistency.

Graph databases

Map Reduce against text files

Java and Python.

MySQLHandlerSocketMemcache API, NDB API

BLOBSELECT v FROM ... WHERE k=?

Functional indexesVirtual columns, etc...

Synchronous replicationGalera, NDB

Damn N:N relations!

Map Reduce against RDBMS

C++ can be done rightDrizzle

Page 25: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 25

Clayton Christensen on Disruptive innovation

Page 26: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 26

Clayton Christensen on Disruptive innovation

First they ignore you

Then they laugh at you

Then they fight you

Then you win

- Gandhi

Page 27: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 27

It used to be

Page 28: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 28

The future is

Page 29: Froscon2011: How i learned to use sql and then learned not to use it

2011-07-26 OSCON 2011 29

The future is

All Open SourceAll Open Source


STAFF SYMPOSIUM SERIES · SQL – Structured Query Language Find table and field names to use in my SQL statements? What applications can use to make SQL statements? Basic SQL Commands

Oracle SQL Internals Handbook - Lagout SQL...Oracle SQL Internals Handbook ... SQL Index Tuning Chapter 9 ... (dba_tables, user_indexes). SQL – All SQL is formatted for easy use

Parallel Query Execution in SQL Server · –On SQL Server 2005, use sys.dm_os_waiting_tasks –On SQL Server 2000, use sysprocesses • All but a few rare cases fixed in SQL Server

Use a Cabeça! SQL - Portugues

MT32 How relational (SQL) and unstructured data (Hadoop) learned to get along

Use a Cabeça SQL - Portugues.pdf

How to use Proc SQL - sasCommunity€¦ ·  · 2017-04-11How to use Proc SQL -1- How to use Proc SQL Tasha Chapman, Oregon Department of Consumer and Business Services SQL1 is most

PL SQL Use Full Programs

When to use Node? Lessons learned

SQL IN PL/SQL · In PL/SQL, there are two main methods of giving value to variables in a PL/SQL block. The first one, which you learned in Chapter 2, “PL/SQL ... because of the

Why Use SQL Safe Backup?

Enterprise Use Cases for No Sql

Faster T-SQL sprocs and functions: Lessons Learned T-SQL sprocs and functions: Lessons Learned ... ALTER INDEX REBUILD or DBCC DBREINDEX ... Since SQL Server 2005 SP3,

Standardize Your Data Preparation in SAS: Use SQL! May 2009 · Standardize Your Data Preparation in SAS: Use SQL! May 2009 Standardize Your Data Preparation in SAS: Use SQL! u 1

SQL*Plus Quick ReferenceExecutes a host operating system command without leaving SQL*Plus. Manipulating SQL, SQL*Plus, and PL/SQL Commands Use the following commands to edit SQL commands

Profiling PL/SQL for Performance - Proligence · PL/SQL for Performance Arup Nanda ... • Use PL/SQL Tracing to trace the lines ... • Use Tracing to identify exceptions Profiling

SQL continued CMSC 461 Michael Wilson. Right back into it Recap: Learned how to log in to PostgreSQL Learned about PostgreSQL data types Learned

Lessons Learned: Case Studies in Sustainable Use Learned: Case Studies in Sustainable Use Factors Influencing the Sustainability of Resource Use and Management Within Multiple Use

SQLCAT: SQL Server 2012 AlwaysOn Lessons Learned from Early

SQL (Use a Cabeça)

Use a Cabeça - SQL

Real-Life SQL Server 2012: AlwaysOn Lessons Learned · Real-Life SQL Server 2012: AlwaysOn Lessons Learned Brent Ozar: MCM, MVP, ... AlwaysOn AGs 5-minute intro ... Failover unit

sql - What I Learned Today...web2learning.net/wp-content/uploads/2009/04/koha-sql.pdfQuerying SQL Once you know your database structure, querying SQL is very straightforward. For Example:

Livro - Use a Cabeça - SQLLivro - Use a Cabeça - SQL

Preparing to Use SQL Server 2000

Database Processing · • To create SQL queries that use the SQL logical operators including AND, OR, and NOT • To create SQL queries that use the SQL built-in aggregate functions

Faster T-SQL sprocs and functions: Lessons Learned

Why use PL/SQL? - Oracle

SQL Packager 5 - Redgatedownloads.red-gate.com/.../SQL_Packager_5_5.pdf · SQL Packager 5.5 July 2008 ... You can use SQL Packager to package SQL Server 2008, SQL Server 2005,

What I Learned About SQL Server at Ignite 2015

Challenges and Lessons Learned Using Blaise IS with SQL Server

JSON SQL Injection and the Lessons Learned

Database Programming with SQL - Ramos' Blog · PDF file•You've learned the syntax rules for generating a SQL query ... In this lesson, you should have learned how to: ... Oracle

Useful Sql Stuff wot I learned

Use a cabeça sql