4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 1/22

rqliteReplicating SQLite using Raft consensus

Philip O'TooleGoSF April 20th 2016

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 2/22

About me

Director of Data Platform Engineering at Percolate.

Previously a full-time core developer with In�uxDB.

Led the team that built Loggly's 2nd generation indexing and search platform.

Big fan of Go, databases, and distributed systems.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 3/22

About rqlite

rqlite is a distributed system, which fully replicates a SQLite database using the Raftconsensus protocol.

You can �nd the source at github.com/otoolep/rqlite (https://github.com/otoolep/rqlite) .

The current release is v2.2.1.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 4/22

Why?

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 5/22

Why replicate SQLite?

Rock solid relational database, within a single C source �le.

With replication, you get reliability.

Easy installation and deployment, thanks to Go.

Lightweight operation.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 6/22

What is Raft?

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 7/22

Raft is a distributed consensus protocol.

Such protocols are used to ensure multiple di�erent nodes -- servers -- always agreeon a given set of values.

It allows us to build a cluster of servers, such that for a quorum of servers within thecluster, each of those servers has the same state.

Within rqlite that state is a SQLite database.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 8/22

rqlite architecture

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 9/22

A rqlite node

┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ┐ Clients Other └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ │ Nodes │ │ ─ ─ ─ ─ ─ │ ▲ │ │ │ │ ▼ ▼ ┌─────────────────────────────┐ ┌───────────────┐ │ HTTP(S) │ │ TCP │ └─────────────────────────────┘ └───────────────┘ ┌───────────────────────────────────────────────┐ │ Raft (hashicorp/raft) │ └───────────────────────────────────────────────┘ ┌───────────────────────────────────────────────┐ │ matt-n/go-sqlite3 │ └───────────────────────────────────────────────┘ ┌───────────────────────────────────────────────┐ │ sqlite3.c │ └───────────────────────────────────────────────┘

All repositories hosted on Github

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 10/22

rqlite clusters

rqlite clusters are most practical when 3, 5, or 7 nodes in size.

Even numbers don't get you anything, and larger than 9 becomes unwieldy.

rqlite is not suitable for replicating massive clusters, such as all the SQLite databaseson a smartphone network.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 11/22

Hashicorp Raft

Raft implementation from Hashicorp, available at github.com/hashicorp/raft(https://github.com/hashicorp/raft)

Used by Consul, Nomad, and In�uxDB.

Another well known Raft implementation has been written by CoreOS, available atgithub.com/coreos/etcd/tree/master/raft (https://github.com/coreos/etcd/tree/master/raft)

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 12/22

Integrating with Hashicorp Raft

Integrating with the Raft consensus module involves implementing �ve key functions.

Apply(l *raft.Log) interface{} // Apply a committed entry to the state machine Snapshot() (raft.FSMSnapshot, error) // Returns a snapshot of the state machine Restore(rc io.ReadCloser) error // Create state machine from snapshot Persist(sink raft.SnapshotSink) // Write snapshot to persistent storage Release() // Snapshot release. Usually a no-op

A reference use of Hashicorp's Raft implementation is available atgithub.com/otoolep/hraftd (https://github.com/otoolep/hraftd)

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 13/22

Demo

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 14/22

Lessons learned

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 15/22

"Plan to throw one away. You will, anyhow."

rqlite v1.0 su�ered from many shortcomings.

Poor API.

Used the generic Go SQL interfaces.

Non-idiomatic Go code.

go-raft as the consensus module, which was nearing its end-of-life.

But v1.0 was still used to build github.com/openmarket/openmarket(https://github.com/openmarket/openmarket) , a bitcoin-based marketplace!

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 16/22

rqlite v2.2.1

Idiomatic Go code.

Much better API, thanks to experience with In�uxDB.

Limited use of 3rd party libraries.

Good use of interfaces, which allows better testing.

Direct use of the Go SQLite package.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 17/22

Better code means better features

Read consistency levels: strong, weak, and none.

Hot backups.

In-memory database support.

User-level permissioning.

There is now a Python driver available at github.com/zmedico/pyrqlite(https://github.com/zmedico/pyrqlite)

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 18/22

Next steps

rqlite is software, and software is never �nished.

Proper dependency management.

A fully-featured CLI.

Leader redirection.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 19/22

What rqlite can do

With it you've got a lightweight and reliable distributed store for relational data.

You could use rqlite as part of a larger system, as a central store for some criticalrelational data, without having to run a heavier solution like MySQL.

rqlite might also be an e�ective way to provide a small number of SQLite read-replicas.

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 20/22

References

github.com/otoolep/rqlite (https://github.com/otoolep/rqlite)

github.com/hashicorp/raft (https://github.com/hashicorp/raft)

github.com/mattn/go-sqlite3 (https://github.com/mattn/go-sqlite3)

github.com/otoolep/hraftd (https://github.com/otoolep/hraftd)

github.com/zmedico/pyrqlite (https://github.com/zmedico/pyrqlite)

github.com/goraft/raft (https://github.com/goraft/raft)

github.com/openmarket/openmarket (https://github.com/openmarket/openmarket)

raft.github.io/ (https://raft.github.io/)

thesecretlivesofdata.com/raft/ (http://thesecretlivesofdata.com/raft/)

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 21/22

Thank you

Philip O'TooleGoSF April 20th 2016

http://www.philipotoole.com/ (http://www.philipotoole.com/)

@general_order24 (http://twitter.com/general_order24)

4/20/2016 rqlite

http://127.0.0.1:3999/rqlite.slide#17 22/22