bigchaindb documentation · chapter 1 table of contents 1.1introduction bigchaindb is a scalable...

31
BigchainDB Documentation Release 0.1.4 BigchainDB Contributors April 07, 2016

Upload: others

Post on 02-Aug-2020

14 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB DocumentationRelease 0.1.4

BigchainDB Contributors

April 07, 2016

Page 2: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture
Page 3: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

Contents

1 Table of Contents 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Installing and Running BigchainDB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2.1 Install and Run RethinkDB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.2 Install Python 3.4+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.3 Install BigchainDB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

How to Install BigchainDB with pip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2How to Install BigchainDB from Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3How to Install BigchainDB on a VM with Vagrant . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.4 Run BigchainDB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.5 Run BigchainDB with Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Running Unit Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3.1 Using docker-compose to Run the Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 The Python Server API by Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4.1 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.2 Create a Digital Asset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4.3 Read the Creation Transaction from the DB . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.4 Transfer the Digital Asset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4.5 Double Spends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.5 The BigchainDB Command Line Interfaces (CLIs) . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.6 The HTTP Client-Server API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.7 The Python Driver API by Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.8 Deploying a Local Multi-Node RethinkDB Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.8.1 Launching More RethinkDB Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.8.2 Securing the Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.8.3 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.9 Deploy a Cluster on AWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.9.1 Why? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.9.2 Python Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.9.3 AWS Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Create an AWS Access Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Get Enough Amazon Elastic IP Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Create an Amazon EC2 Key Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Create an Amazon EC2 Security Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.9.4 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.9.5 Known Deployment Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

NetworkError . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Failure of sudo apt-get update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

i

Page 4: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

Failure when Installing Base Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.10 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.10.1 Hashes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.10.2 Signature algorithm and keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.11 The Transaction, Block and Vote Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.11.1 The Transaction Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.11.2 The Block Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.11.3 The Vote Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.12 JSON Serialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.13 Developer Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.14 BigchainDB Consensus Plugins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.14.1 Installing a plugin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.14.2 Plugin API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.14.3 Extending BigchainDB behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.14.4 Packaging a plugin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

1.15 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.16 Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.17 Contributing to the BigchainDB Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.18 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231.19 Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2 Indices and Tables 25

ii

Page 5: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

CHAPTER 1

Table of Contents

1.1 Introduction

BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecturein the BigchainDB whitepaper. This document, the BigchainDB Documentation, is intended primarily for:

1. Developers of BigchainDB server software.

2. People deploying and managing BigchainDB clusters.

3. Developers of BigchainDB driver software (SDKs used to develop client software).

4. App developers who are developing client apps to talk to one or more live, operational BigchainDB clusters.They would use one of the BigchainDB drivers.

If you’re curious about what’s in our roadmap, see the ROADMAP.md file and the list of open issues. If you want torequest a feature, file a bug report, or make a pull request, see the CONTRIBUTING.md file.

1.2 Installing and Running BigchainDB Server

We’re developing BigchainDB Server on Ubuntu 14.04, but it should work on any OS that runs RethinkDB Server andPython 3.4+. (BigchainDB Server is built on top of RethinkDB Server.)

BigchainDB Server is intended to be run on each server in a large distributed cluster of servers; it’s not very usefulrunning by itself on a single computer. That’s why we’re developing it on Ubuntu 14.04: it’s one of the more commonserver operating systems.

Mac OS X users may get the best results running BigchainDB Server with Docker.

Windows users may get the best results running BigchainDB Server in a VM with Vagrant.

(BigchainDB clients should run on a much larger array of operating systems.)

1.2.1 Install and Run RethinkDB Server

If you don’t already have RethinkDB Server installed on your server, you must install it. The RethinkDB documenta-tion has instructions for how to install RethinkDB Server on a variety of operating systems.

RethinkDB Server doesn’t require any special configuration. You can run it by opening a Terminal and entering:

$ rethinkdb

1

Page 6: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.2.2 Install Python 3.4+

If you don’t already have it, then you should install Python 3.4+ (maybe in a virtual environment, so it doesn’t conflictwith other Python projects you’re working on).

1.2.3 Install BigchainDB Server

BigchainDB Server has some OS-level dependencies. In particular, you need to install the OS-level dependencies forthe Python cryptography package. Instructions for installing those dependencies on your OS can be found in thecryptography package documentation.

On Ubuntu 14.04, we found that the following was enough:

$ sudo apt-get update$ sudo apt-get install libffi-dev g++ libssl-dev python3-dev

On Fedora 23, we found that the following was enough (tested in February 2015):

$ sudo dnf update$ sudo dnf install libffi-devel gcc-c++ redhat-rpm-config python3-devel openssl-devel

(If you’re using a version of Fedora before version 22, you may have to use yum instead of dnf.)

With OS-level dependencies installed, you can install BigchainDB Server with pip or from source.

How to Install BigchainDB with pip

BigchainDB (i.e. both the Server and the officially-supported drivers) is distributed as a Python package on PyPI soyou can install it using pip. First, make sure you have a version of pip installed for Python 3.4+:

$ pip -V

If it says that pip isn’t installed, or it says pip is associated with a Python version less than 3.4, then you must installa pip version associated with Python 3.4+. See the pip installation instructions. On Ubuntu 14.04, we found thatthis works:

$ sudo apt-get install python3-setuptools$ sudo easy_install3 pip

(Note: Using sudo apt-get python3-pip also installs a Python 3 version of pip (named pip3) but we foundit installed a very old version and there were issues with updating it.)

Once you have a version of pip associated with Python 3.4+, then you can install BigchainDB Server (and officially-supported BigchainDB drivers) using:

sudo pip install bigchaindb

(or maybe sudo pip3 install bigchaindb or sudo pip3.4 install bigchaindb. The sudo maynot be necessary.)

Note: You can use pip to upgrade the bigchaindb package to the latest version using sudo pip install--upgrade bigchaindb

2 Chapter 1. Table of Contents

Page 7: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

How to Install BigchainDB from Source

BigchainDB (i.e. both the Server and the officially-supported drivers) is in its early stages and being actively developedon its GitHub repository. Contributions are highly appreciated. If you want to help with development, then you’ll wantto install BigchainDB from source. Here’s how.

First, clone the public repository:

$ git clone [email protected]:bigchaindb/bigchaindb.git

Install from the source:

$ python setup.py install

How to Install BigchainDB on a VM with Vagrant

One of our community members (@Mec-Is) wrote a page about how to install BigchainDB on a VM with Vagrant.

1.2.4 Run BigchainDB Server

Once you’ve installed BigchainDB Server, you can run it. First make sure you have RethinkDB running:

$ rethinkdb

Then open a different terminal and run:

$ bigchaindb start

During its first run, BigchainDB Server takes care of configuring a single node environment.

1.2.5 Run BigchainDB with Docker

NOT for Production Use

For those who like using Docker and wish to experiment with BigchainDB in non-production environments,we currently maintain a dockerfile that can be used to build an image for bigchaindb, along with adocker-compose.yml file to manage a “standalone node”, consisting mainly of two containers: one for Re-thinkDB, and another for BigchainDB.

Assuming you have docker and docker-compose installed, you would proceed as follows.

In a terminal shell:

$ git clone [email protected]:bigchaindb/bigchaindb.git

Build the Docker image:

$ docker-compose build

then do a one-time configuration step to create the config file; it will be stored on your host machine under~/.bigchaindb_docker/config:

$ docker-compose run --rm bigchaindb bigchaindb configure

You can then start it up (in the background, as a daemon) using:

1.2. Installing and Running BigchainDB Server 3

Page 8: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

$ docker-compose up -d

then you can load test transactions via:

$ docker-compose run --rm bigchaindb bigchaindb-benchmark load

If you’re on Linux, you can probably view the RethinkDB dashboard at:

http://localhost:58080/

If that doesn’t work, then replace localhost with the IP or hostname of the machine running the Docker engine. Ifyou are running docker-machine (e.g.: on Mac OS X) this will be the IP of the Docker machine (docker-machineip machine_name).

1.3 Running Unit Tests

Once you’ve installed BigchainDB Server, you may want to run all the unit tests. This section explains how.

First of all, if you installed BigchainDB Server using pip (i.e. by getting the package from PyPI), then you didn’tinstall the tests. Before you can run all the unit tests, you must install BigchainDB from source.

To run all the unit tests, first make sure you have RethinkDB running:

$ rethinkdb

then in another terminal, do:

$ py.test -v

If the above command doesn’t work (e.g. maybe you are running in a conda virtual environment), try:

$ python -m pytest -v

(We write our unit tests using the pytest framework.)

You can also run all unit tests via setup.py, using:

$ python setup.py test

1.3.1 Using docker-compose to Run the Tests

You can also use docker-compose to run the unit tests. (You don’t have to start RethinkDB first:docker-compose does that on its own, when it reads the docker-compose.yml file.)

First, build the images (~once), using:

$ docker-compose build

then run the unit tests using:

$ docker-compose run --rm bigchaindb py.test -v

1.4 The Python Server API by Example

This section gives an example of using the Python Server API to interact directly with a BigchainDB node runningBigchainDB Server. That is, in this example, the Python code and BigchainDB Server run on the same machine.

4 Chapter 1. Table of Contents

Page 9: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

(One can also interact with a BigchainDB node via other APIs, including the HTTP Client-Server API.)

We create a digital asset, sign it, write it to a BigchainDB Server instance, read it, transfer it to a different user, andthen attempt to transfer it to another user, resulting in a double-spend error.

1.4.1 Getting Started

First, make sure you have RethinkDB and BigchainDB installed and running, i.e. you installed them and you ran:

$ rethinkdb$ bigchaindb start

Don’t shut them down! In a new terminal, open a Python shell:

$ python

Now we can import the Bigchain class and create an instance:

from bigchaindb import Bigchainb = Bigchain()

This instantiates an object b of class Bigchain. When instantiating a Bigchain object without arguments (asabove), it reads the configurations stored in $HOME/.bigchaindb.

In a federation of BigchainDB nodes, each node has its own Bigchain instance.

The Bigchain class is the main API for all BigchainDB interactions, right now. It does things that BigchainDBnodes do, but it also does things that BigchainDB clients do. In the future, it will be broken apart into a node/serverclass and a client class.

The Bigchain class is documented in the Developer Interface section.

1.4.2 Create a Digital Asset

At a high level, a “digital asset” is something which can be represented digitally and can be assigned to a user. InBigchainDB, users are identified by their public key, and the data payload in a digital asset is represented using ageneric Python dict.

In BigchainDB, only the federation nodes are allowed to create digital assets, by doing a special kind of transaction: aCREATE transaction.

from bigchaindb import crypto

# create a test usertestuser1_priv, testuser1_pub = crypto.generate_key_pair()

# define a digital asset data payloaddigital_asset_payload = {'msg': 'Hello BigchainDB!'}

# a create transaction uses the operation `CREATE` and has no inputstx = b.create_transaction(b.me, testuser1_pub, None, 'CREATE', payload=digital_asset_payload)

# all transactions need to be signed by the user creating the transactiontx_signed = b.sign_transaction(tx, b.me_private)

# write the transaction to the bigchain# the transaction will be stored in a backlog where it will be validated,

1.4. The Python Server API by Example 5

Page 10: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

# included in a block, and written to the bigchainb.write_transaction(tx_signed)

1.4.3 Read the Creation Transaction from the DB

After a couple of seconds, we can check if the transactions was included in the bigchain:

# retrieve a transaction from the bigchaintx_retrieved = b.get_transaction(tx_signed['id'])

'id': '6539dded9479c47b3c83385ae569ecaa90bcf387240d1ee2ea3ae0f7986aeddd','transaction': { 'current_owner': 'pvGtcm5dvwWMzCqagki1N6CDKYs2J1cCwTNw8CqJic3Q',

'data': { 'hash': '872fa6e6f46246cd44afdb2ee9cfae0e72885fb0910e2bcf9a5a2a4eadb417b8','payload': {'msg': 'Hello BigchainDB!'}},

'input': None,'new_owner': 'ssQnnjketNYmbU3hwgFMEQsc4JVYAmZyWHnHCtFS8aeA','operation': 'CREATE','timestamp': '1455108421.753908'}}

The new owner of the digital asset is now ssQnnjketNYmbU3hwgFMEQsc4JVYAmZyWHnHCtFS8aeA, which isthe public key of testuser1.

1.4.4 Transfer the Digital Asset

Now that testuser1 has a digital asset assigned to him, he can transfer it to another user. Transfer transactionsrequire an input. The input will be the transaction id of a digital asset that was assigned to testuser1, which in ourcase is 6539dded9479c47b3c83385ae569ecaa90bcf387240d1ee2ea3ae0f7986aeddd.

# create a second testusertestuser2_priv, testuser2_pub = b.generate_keys()

# create a transfer transactiontx_transfer = b.create_transaction(testuser1_pub, testuser2_pub, tx_retrieved['id'], 'TRANSFER')

# sign the transactiontx_transfer_signed = b.sign_transaction(tx_transfer, testuser1_priv)

# write the transactionb.write_transaction(tx_transfer_signed)

# check if the transaction is already in the bigchaintx_transfer_retrieved = b.get_transaction(tx_transfer_signed['id'])

{ 'id': '1b78c313257540189f27da480152ed8c0b758569cdadd123d9810c057da408c3','signature': '3045022056166de447001db8ef024cfa1eecdba4306f92688920ac24325729d5a5068d47022100fbd495077cb1040c48bd7dc050b2515b296ca215cb5ce3369f094928e31955f6','transaction': { 'current_owner': 'ssQnnjketNYmbU3hwgFMEQsc4JVYAmZyWHnHCtFS8aeA',

'data': None,'input': '6539dded9479c47b3c83385ae569ecaa90bcf387240d1ee2ea3ae0f7986aeddd','new_owner': 'zVzophT73m4Wvf3f8gFYokddkYe3b9PbaMzobiUK7fmP','operation': 'TRANSFER','timestamp': '1455109497.480323'}}

6 Chapter 1. Table of Contents

Page 11: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.4.5 Double Spends

BigchainDB makes sure that a user can’t transfer the same digital asset two or more times (i.e. it prevents doublespends).

If we try to create another transaction with the same input as before, the transaction will be marked invalid and thevalidation will throw a double spend exception:

# create another transfer transaction with the same inputtx_transfer2 = b.create_transaction(testuser1_pub, testuser2_pub, tx_retrieved['id'], 'TRANSFER')

# sign the transactiontx_transfer_signed2 = b.sign_transaction(tx_transfer2, testuser1_priv)

# check if the transaction is validb.validate_transaction(tx_transfer_signed2)Exception: input `6539dded9479c47b3c83385ae569ecaa90bcf387240d1ee2ea3ae0f7986aeddd` was already spent

1.5 The BigchainDB Command Line Interfaces (CLIs)

BigchainDB has some Command Line Interfaces (CLIs). One of them is the bigchaindb command which wealready saw when we first started BigchainDB using:

$ bigchaindb start

The fist time you run bigchaindb start, it creates a default configuration file in $HOME/.bigchaindb. Youcan check that configuration using:

$ bigchaindb show-config

To find out what else you can do with the bigchain command, use:

$ bigchaindb -h

There’s another command named bigchaindb-benchmark. It’s used to run benchmarking tests. You can learnmore about it using:

$ bigchaindb-benchmark -h$ bigchaindb-benchmark load -h

1.6 The HTTP Client-Server API

When you start Bigchaindb using bigchaindb start, an HTTP API is exposed at:

http://localhost:5000/api/v1/

Right now, that API can only be accessed from localhost (i.e. not remotely). In the future, we’ll enable remote accessand explain how that works. See Issue #149 on GitHub.

The HTTP API currently exposes two endpoints, one to get information about a specific transaction id, and one topush a transaction to the BigchainDB cluster. Those endpoints are documented at:

http://docs.bigchaindb.apiary.io/

1.5. The BigchainDB Command Line Interfaces (CLIs) 7

Page 12: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.7 The Python Driver API by Example

The Python driver API is used by app developers to develop client apps which can communicate with one or moreBigchainDB clusters. Under the hood, the Python driver API communicates with the BigchainDB cluster using theBigchainDB HTTP client-server API.

Here’s an example of how to use the Python driver API. First, launch an interactive Python session, then:

In [1]: from bigchaindb.client import temp_clientIn [2]: c1 = temp_client()In [3]: c2 = temp_client()In [4]: tx1 = c1.create()In [5]: tx1Out[5]:{'assignee': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr','id': '26f21d8b5f9731cef631733b8cd1da05f87aa59eb2f939277a2fefeb774ae133','signature': '304402201b904f22e9f5a502070244b64822adf28...','transaction': {'current_owner': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr','data': {'hash': 'efbde2c3aee204a69b7696d4b10ff31137fe78e3946306284f806e2dfc68b805','payload': None},

'input': None,'new_owner': '247epGEcoX9m6yvR6sEZvYGb1XCpUUWtCNUVKgJGrFWCr','operation': 'CREATE','timestamp': '1456763521.824126'}}

In [7]: c1.transfer(c2.public_key, tx1['id'])Out[7]:{'assignee': '2Bi5NUv1UL7h3ZGs5AsE6Gr3oPQhE2vGsYCapNYrAU4pr','id': '34b62c9fdfd93f5907f35e2495239ae1cb62e9519ff64a8710f3f77a9f040857','signature': '3046022100b2b2432c20310dfcda6a2bab3c893b0cd17e70fe...','transaction': {'current_owner': '247epGEcoX9m6yvR6sEZvYGb1XCpUUWtCNUVKgJGrFWCr','data': {'hash': 'efbde2c3aee204a69b7696d4b10ff31137fe78e3946306284f806e2dfc68b805','payload': None},

'input': '26f21d8b5f9731cef631733b8cd1da05f87aa59eb2f939277a2fefeb774ae133','new_owner': 'p5Ci1KJkPHvRBnxqyq36m8GXwkWSuhMiZSg8aB1ZrZgJ','operation': 'TRANSFER','timestamp': '1456763549.446138'}}

1.8 Deploying a Local Multi-Node RethinkDB Cluster

This section explains one way to deploy a multi-node RethinkDB cluster on one machine. You could use such a clusteras a storage backend for a BigchaindB process running on that machine.

1.8.1 Launching More RethinkDB Nodes

Assuming you’ve already installed RethinkDB and have one RethinkDB node running, here’s how you can launch twomore nodes on the same machine. First, prepare two additional nodes. Note that the user who starts RethinkDB musthave write access to the created directories:

mkdir -p /path/to/node2mkdir -p /path/to/node3

then launch two more nodes:

8 Chapter 1. Table of Contents

Page 13: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

rethinkdb --port-offset 1 --directory /path/to/node2 --join localhost:29015rethinkdb --port-offset 2 --directory /path/to/node3 --join localhost:29015

You should now have a three-node RethinkDB cluster up and running. You should be able to monitor it athttp://localhost:8080.

Note: if you play around with replication, the number of transactions will increase. For the real throughput, you shoulddivide the number of transactions by the number of replicas.

1.8.2 Securing the Cluster

We have turned on the bind=all option for connecting other nodes and making RethinkDB accessible from outsidethe server. Unfortunately, that is insecure, so we need to block access to the RethinkDB cluster from the Internet. Atthe same time, we need to allow access to its services from authorized computers.

For the cluster port, we will use a firewall to enclose our cluster. For the web management console and the driver port,we will use SSH tunnels to access them from outside the server. SSH tunnels redirect requests on a client computerto a remote computer over SSH, giving the client access to all of the services only available on the remote server’slocalhost name space.

Repeat the following steps on all your RethinkDB servers.

First, block all outside connections:

# the web management consolesudo iptables -A INPUT -i eth0 -p tcp --dport 8080 -j DROPsudo iptables -I INPUT -i eth0 -s 127.0.0.1 -p tcp --dport 8080 -j ACCEPT

# the driver portsudo iptables -A INPUT -i eth0 -p tcp --dport 28015 -j DROPsudo iptables -I INPUT -i eth0 -s 127.0.0.1 -p tcp --dport 28015 -j ACCEPT

# the communication portsudo iptables -A INPUT -i eth0 -p tcp --dport 29015 -j DROPsudo iptables -I INPUT -i eth0 -s 127.0.0.1 -p tcp --dport 29015 -j ACCEPT

Save the iptables config:

sudo apt-get updatesudo apt-get install iptables-persistent

You can find out more about iptables in the man pages.

1.8.3 Troubleshooting

You can get help with RethinkDB from RethinkDB experts and the RethinkDB community.

If your nodes in a sharded and replicated cluster are not in sync, it may help if you delete the data of the affected nodeand restart the node. If there are no other problems, your node will come back and sync within a couple of minutes.You can verify this by monitoring the cluster via the monitoring url: http://localhost:8080.

1.9 Deploy a Cluster on AWS

This section explains a way to deploy a cluster of BigchainDB nodes on Amazon Web Services (AWS). We use someBash and Python scripts to launch several instances (virtual servers) on Amazon Elastic Compute Cloud (EC2). Then

1.9. Deploy a Cluster on AWS 9

Page 14: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

we use Fabric to install RethinkDB and BigchainDB on all those instances.

NOTE: At the time of writing, these script do launch a bunch of EC2 instances, and they do install RethinkDBplus BigchainDB on each instance, but don’t expect to be able to use the cluster for anything useful. There areseveral issues related to configuration, networking, and external clients that must be sorted out first. That said,you might find it useful to try out the AWS deployment scripts, because setting up to use them, and using them,will be very similar once those issues get sorted out.

1.9.1 Why?

You might ask why one would want to deploy a centrally-controlled BigchainDB cluster. Isn’t BigchainDB supposedto be decentralized, where each node is controlled by a different person or organization?

That’s true, but there are some reasons why one might want a centrally-controlled cluster: 1) for testing, and 2) forinitial deployment. Afterwards, the control of each node can be handed over to a different entity.

1.9.2 Python Setup

The instructions that follow have been tested on Ubuntu 14.04, but may also work on similar distros or operatingsystems.

Note: Our Python scripts for deploying to AWS use Python 2 because Fabric doesn’t work with Python 3.

Maybe create a Python 2 virtual environment and activate it. Then install the following Python packages (in that virtualenvironment):

pip install fabric fabtools requests boto3 awscli

What did you just install?

• “Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for applicationdeployment or systems administration tasks.”

• fabtools are “tools for writing awesome Fabric files”

• requests is a Python package/library for sending HTTP requests

• “Boto is the Amazon Web Services (AWS) SDK for Python, which allows Python developers to write softwarethat makes use of Amazon services like S3 and EC2.” (boto3 is the name of the latest Boto package.)

• The aws-cli package, which is an AWS Command Line Interface (CLI).

1.9.3 AWS Setup

Before you can deploy a BigchainDB cluster on AWS, you must have an AWS account. If you don’t already have one,you can sign up for one for free.

Create an AWS Access Key

The next thing you’ll need is an AWS access key. If you don’t have one, you can create one using the instructions inthe AWS documentation. You should get an access key ID (e.g. AKIAIOSFODNN7EXAMPLE) and a secret accesskey (e.g. wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY).

You should also pick a default AWS region name (e.g. eu-central-1). That’s where your cluster will run. TheAWS documentation has a list of them.

10 Chapter 1. Table of Contents

Page 15: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

Once you’ve got your AWS access key, and you’ve picked a default AWS region name, go to a terminal session andenter:

aws configure

and answer the four questions. For example:

AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLEAWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEYDefault region name [None]: eu-central-1Default output format [None]: [Press Enter]

This writes two files:

• ~/.aws/credentials

• ~/.aws/config

AWS tools and packages look for those files.

Get Enough Amazon Elastic IP Addresses

Our AWS deployment scripts use elastic IP addresses (although that may change in the future). By default, AWSaccounts get five elastic IP addresses. If you want to deploy a cluster with more than five nodes, then you will needmore than five elastic IP addresses; you may have to apply for those; see the AWS documentation on elastic IPaddresses.

Create an Amazon EC2 Key Pair

Go to the AWS EC2 Console and select “Key Pairs” in the left sidebar. Click the “Create Key Pair” but-ton. Give it the name bigchaindb. You should be prompted to save a file named bigchaindb.pem.That file contains the RSA private key. (Amazon keeps the corresponding public key.) Save the file inbigchaindb/deploy-cluster-aws/pem/bigchaindb.pem.

You should not share your private key.

Create an Amazon EC2 Security Group

Go to the AWS EC2 Console and select “Security Groups” in the left sidebar. Click the “Create Security Group”button. Give it the name bigchaindb. The description probably doesn’t matter but we also put bigchaindb forthat.

Add some rules for Inbound traffic:

• Type = All TCP, Protocol = TCP, Port Range = 0-65535, Source = 0.0.0.0/0

• Type = SSH, Protocol = SSH, Port Range = 22, Source = 0.0.0.0/0

• Type = All UDP, Protocol = UDP, Port Range = 0-65535, Source = 0.0.0.0/0

• Type = All ICMP, Protocol = ICMP, Port Range = 0-65535, Source = 0.0.0.0/0

Note: These rules are extremely lax! They’re meant to make testing easy. You’ll want to tighten them up if youintend to have a secure cluster. For example, Source = 0.0.0.0/0 is CIDR notation for “allow this traffic to come fromany IP address.”

1.9. Deploy a Cluster on AWS 11

Page 16: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.9.4 Deployment

Here’s an example of how one could launch a BigchainDB cluster of 4 nodes tagged wrigley on AWS:

cd bigchaindbcd deploy-cluster-aws./startup.sh wrigley 4

startup.sh is a Bash script which calls some Python 2 and Fabric scripts. Here’s what it does:

1. allocates more elastic IP addresses if necessary,

2. launches the specified number of nodes (instances) on Amazon EC2,

3. tags them with the specified tag,

4. waits until those instances exist and are running,

5. for each instance, it associates an elastic IP address with that instance,

6. adds remote keys to ~/.ssh/known_hosts,

7. (re)creates the RethinkDB configuration file conf/rethinkdb.conf,

8. installs base (prerequisite) software on all instances,

9. installs RethinkDB on all instances,

10. installs BigchainDB on all instances,

11. generates the genesis block,

12. starts BigchainDB on all instances.

It should take a few minutes for the deployment to finish. If you run into problems, see the section on KnownDeployment Issues below.

The EC2 Console has a section where you can see all the instances you have running on EC2. You can ssh into arunning instance using a command like:

ssh -i pem/bigchaindb.pem [email protected]

except you’d replace the ec2-52-29-197-211.eu-central-1.compute.amazonaws.com with the pub-lic DNS name of the instance you want to ssh into. You can get that from the EC2 Console: just click on an instanceand look in its details pane at the bottom of the screen. Some commands you might try:

ip addr showsudo service rethinkdb statusbigchaindb --helpbigchaindb show-config

There are fees associated with running instances on EC2, so if you’re not using them, you should terminate them. Youcan do that from the AWS EC2 Console.

The same is true of your allocated elastic IP addresses. There’s a small fee to keep them allocated if they’re notassociated with a running instance. You can release them from the AWS EC2 Console.

1.9.5 Known Deployment Issues

NetworkError

If you tested with a high sequence it might be possible that you run into an error message like this:

12 Chapter 1. Table of Contents

Page 17: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

NetworkError: Host key for ec2-xx-xx-xx-xx.eu-central-1.compute.amazonaws.comdid not match pre-existing key! Server's key was changed recently, or possibleman-in-the-middle attack.

If so, just clean up your known_hosts file and start again. For example, you might copy your currentknown_hosts file to old_known_hosts like so:

mv ~/.ssh/known_hosts ~/.ssh/old_known_hosts

Then terminate your instances and try deploying again with a different tag.

Failure of sudo apt-get update

The first thing that’s done on all the instances, once they’re running, is basically sudo apt-get update. Some-times that fails. If so, just terminate your instances and try deploying again with a different tag. (These problems seemto be time-bounded, so maybe wait a couple of hours before retrying.)

Failure when Installing Base Software

If you get an error with installing the base software on the instances, then just terminate your instances and trydeploying again with a different tag.

1.10 Cryptography

The section documents the cryptographic algorithms and Python implementations that we use.

The implementations that we have chosen for now are just for fast prototyping. Some of them are pure Pythonimplementations which may be slow. As future work, we should look at other alternatives.

1.10.1 Hashes

For hashing we are using the sha3-256 algorithm and pysha3 as the Python implementation. We store the hex encodedhash in the database. For example:

import hashlib# monkey patch hashlib with sha3 functionsimport sha3

data = "message"tx_hash = hashlib.sha3_256(data).hexdigest()

1.10.2 Signature algorithm and keys

The signature algorithm used by BigchainDB is ED25519 using the python ed25519 module, overloaded by the cryp-toconditions library.

The private key is the base58 encoded hexadecimal representation of private number. The public key is the base58encoded hexadecimal representation of the compressed public numbers.

1.10. Cryptography 13

Page 18: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.11 The Transaction, Block and Vote Models

Transactions, blocks and votes are represented using JSON documents with the following models (schemas). See thesection on cryptography for more information about how we calculate hashes and signatures.

1.11.1 The Transaction Model

{"id": "<sha3 hash>","transaction": {

"current_owner": "<pub-key>","new_owner": "<pub-key>","input": "<sha3 hash>","operation": "<string>","timestamp": "<timestamp from client>","data": {

"hash": "<SHA3-256 hash hexdigest of payload>","payload": {

"title": "The Winds of Plast","creator": "Johnathan Plunkett","IPFS_key": "QmfQ5QAjvg4GtA3wg3adpnDJug8ktA1BxurVqBD8rtgVjP"

}}

},"signature": "<signature of the transaction>"

}

A transaction is an operation between the current_owner and the new_owner over the digital content describedby hash. For example if could be a transfer of ownership of the digital content hash

• id: sha3 hash of the transaction. The id is also the DB primary key.

• current_owner: Public key of the current owner of the digital content with hash hash

• new_owner: Public key of the new owner of the digital content with hash hash

• input: id (sha3 hash) of the transaction in which the content was transfered to the user (similar to input in theblockchain). Right now we will assume that there is only one input per transaction to simplify the prototype.This can be changed in the future to allow multiple inputs per transaction.

• operation: String representation of the operation being performed (REGISTER, TRANSFER, ...) this willdefine how the transactions should be validated

• timestamp: Time of creation of the transaction in UTC. It’s provided by the client.

• data: JSON object describing the asset (digital content). It contains at least the field hash which is a sha3hash of the digital content.

• signature: Signature of the transaction with the current_owner private key

1.11.2 The Block Model

{"id": "<sha3 hash of the serialized block contents>","block": {

"timestamp": "<block-creation timestamp>","transactions": ["<list of transactions>"],

14 Chapter 1. Table of Contents

Page 19: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

"node_pubkey": "<public key of the node creating the block>","voters": ["<list of federation nodes public keys>"]

},"signature": "<signature of the block>","votes": ["<list of votes>"]

}

Still to be defined when new blocks are created (after x number of transactions, or after x amount of seconds, or both).

A block contains a group of transactions and includes the hash of the hash of the previous block to build the chain.

• id: sha3 hash of the contents of block (i.e. the timestamp, list of transactions, node_pubkey, and voters). Thisis also a RethinkDB primary key; that’s how we ensure that all blocks are unique.

• block: The actual block

– timestamp: timestamp when the block was created. It’s provided by the node that created the block.

– transactions: the list of transactions included in the block

– node_pubkey: the public key of the node that create the block

– voters: list public keys of the federation nodes. Since the size of the federation may change over timethis will tell us how many nodes existed in the federation when the block was created so that in a laterpoint in time we can check that the block received the correct number of votes.

• signature: Signature of the block by the node that created the block (i.e. To create it, the node serialized theblock contents and signed that with its private key)

• votes: Initially an empty list. New votes are appended as they come in from the nodes.

1.11.3 The Vote Model

Each node must generate a vote for each block, to be appended to that block’s votes list. A vote has the followingstructure:

{"node_pubkey": "<the pubkey of the voting node>","vote": {

"voting_for_block": "<id of the block the node is voting for>","previous_block": "<id of the block previous to this one>","is_block_valid": "<true|false>","invalid_reason": "<None|DOUBLE_SPEND|TRANSACTIONS_HASH_MISMATCH|NODES_PUBKEYS_MISMATCH","timestamp": "<timestamp of the voting action>"

},"signature": "<signature of vote block>"

}

Note: The invalid_reason was not being used as of v0.1.3.

1.12 JSON Serialization

We needed to clearly define how to serialize a JSON object to calculate the hash.

The serialization should produce the same byte output independently of the architecture running the software. If thereare differences in the serialization, hash validations will fail although the transaction is correct.

For example, consider the following two methods of serializing {’a’: 1}:

1.12. JSON Serialization 15

Page 20: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

# Use a serializer provided by RethinkDBa = r.expr({'a': 1}).to_json().run(b.connection)u'{"a":1}'

# Use the serializer in Python's json moduleb = json.dumps({'a': 1})'{"a": 1}'

a == bFalse

The results are not the same. We want a serialization and deserialization so that the following is always true:

deserialize(serialize(data)) == dataTrue

After looking at this further, we decided that the python json module is still the best bet because it complies with theRFC. We can specify the encoding, separators used and enforce it to order by the keys to make sure that we obtainmaximum interoperability.

import json

json.dumps(data, skipkeys=False, ensure_ascii=False,encoding="utf-8", separators=(',', ':'),sort_keys=True)

• skipkeys: With skipkeys False if the provided keys are not a string the serialization will fail. This way weenforce all keys to be strings

• ensure_ascii: The RFC recommends utf-8 for maximum interoperability. By setting ensure_asciito False we allow unicode characters and force the encoding to utf-8.

• separators: We need to define a standard separator to use in the serialization. We did not do this differentimplementations could use different separators for serialization resulting in a still valid transaction but with adifferent hash e.g. an extra whitespace introduced in the serialization would not still create a valid JSON objectbut the hash would be different.

Every time we need to perform some operation on the data like calculating the hash or signing/verifying the transaction,we need to use the previous criteria to serialize the data and then use the byte representation of the serialized data (ifwe treat the data as bytes we eliminate possible encoding errors e.g. unicode characters). For example:

# calculate the hash of a transaction# the transaction is a dictionarytx_serialized = bytes(serialize(tx))tx_hash = hashlib.sha3_256(tx_serialized).hexdigest()

# signing a transactiontx_serialized = bytes(serialize(tx))signature = sk.sign(tx_serialized)

# verify signaturetx_serialized = bytes(serialize(tx))vk.verify(signature, tx_serialized)

16 Chapter 1. Table of Contents

Page 21: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.13 Developer Interface

The Bigchain class is the top-level API for BigchainDB. If you want to create and initialize a BigchainDB database,you create a Bigchain instance (object). Then you can use its various methods to create transactions, write transactions(to the object/database), read transactions, etc.

class bigchaindb.Bigchain(host=None, port=None, dbname=None, public_key=None, pri-vate_key=None, keyring=[], consensus_plugin=None)

Bigchain API

Create, read, sign, write transactions to the database

__init__(host=None, port=None, dbname=None, public_key=None, private_key=None, keyring=[],consensus_plugin=None)

Initialize the Bigchain instance

There are three ways in which the Bigchain instance can get its parameters. The order by which theparameters are chosen are:

1.Setting them by passing them to the __init__ method itself.

2.Setting them as environment variables

3.Reading them from the config.json file.

Parameters

• host (str) – hostname where the rethinkdb is running.

• port (int) – port in which rethinkb is running (usually 28015).

• dbname (str) – the name of the database to connect to (usually bigchain).

• public_key (str) – the base58 encoded public key for the ED25519 curve.

• private_key (str) – the base58 encoded private key for the ED25519 curve.

• keyring (list[str]) – list of base58 encoded public keys of the federation nodes.

create_transaction(*args, **kwargs)Create a new transaction

Refer to the documentation of your consensus plugin.

Returns newly constructed transaction.

Return type dict

sign_transaction(transaction, *args, **kwargs)Sign a transaction

Refer to the documentation of your consensus plugin.

Returns transaction with any signatures applied.

Return type dict

verify_signature(signed_transaction, *args, **kwargs)Verify the signature(s) of a transaction.

Refer to the documentation of your consensus plugin.

Returns

True if the transaction’s required signature data is present and correct, False otherwise.

1.13. Developer Interface 17

Page 22: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

Return type bool

write_transaction(signed_transaction, durability=’soft’)Write the transaction to bigchain.

When first writing a transaction to the bigchain the transaction will be kept in a backlog until it has beenvalidated by the nodes of the federation.

Parameters signed_transaction (dict) – transaction with the signature included.

Returns database response

Return type dict

get_transaction(txid)Retrieve a transaction with txid from bigchain.

Queries the bigchain for a transaction that was already included in a block.

Parameters txid (str) – transaction id of the transaction to query

Returns

A dict with the transaction details if the transaction was found.

If no transaction with that txid was found it returns None

get_tx_by_payload_hash(payload_hash)Retrieves transactions related to a digital asset.

When creating a transaction one of the optional arguments is the payload. The payload is a generic dictthat contains information about the digital asset.

To make it easy to query the bigchain for that digital asset we create a sha3-256 hash of the serializedpayload and store it with the transaction. This makes it easy for developers to keep track of their digitalassets in bigchain.

Parameters payload_hash (str) – sha3-256 hash of the serialized payload.

Returns A list of transactions containing that payload. If no transaction exists with that payloadit returns None

get_spent(txid)Check if a txid was already used as an input.

A transaction can be used as an input for another transaction. Bigchain needs to make sure that a giventxid is only used once.

Parameters txid (str) – transaction id.

Returns The transaction that used the txid as an input if it exists else it returns None

get_owned_ids(owner)Retrieve a list of txids that can we used has inputs.

Parameters owner (str) – base58 encoded public key.

Returns list of txids currently owned by owner

Return type list

validate_transaction(transaction)Validate a transaction.

Parameters transaction (dict) – transaction to validate.

18 Chapter 1. Table of Contents

Page 23: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

Returns The transaction if the transaction is valid else it raises an exception describing the rea-son why the transaction is invalid.

is_valid_transaction(transaction)Check whether a transacion is valid or invalid.

Similar to validate_transaction but does not raise an exception if the transaction is valid.

Parameters transaction (dict) – transaction to check.

Returns True if the transaction is valid, False otherwise

Return type bool

create_block(validated_transactions)Creates a block given a list of validated_transactions.

Note that this method does not validate the transactions. Transactions should be validated before callingcreate_block.

Parameters validated_transactions (list) – list of validated transactions.

Returns created block.

Return type dict

validate_block(block)Validate a block.

Parameters block (dict) – block to validate.

Returns The block if the block is valid else it raises and exception describing the reason whythe block is invalid.

is_valid_block(block)Check whether a block is valid or invalid.

Similar to validate_block but does not raise an exception if the block is invalid.

Parameters block (dict) – block to check.

Returns True if the block is valid, False otherwise.

Return type bool

write_block(block, durability=’soft’)Write a block to bigchain.

Parameters block (dict) – block to write to bigchain.

create_genesis_block()Create the genesis block

Block created when bigchain is first initialized. This method is not atomic, there might be concurrencyproblems if multiple instances try to write the genesis block when the BigchainDB Federation is started,but it’s a highly unlikely scenario.

vote(block, previous_block_id, decision, invalid_reason=None)Cast your vote on the block given the previous_block_hash and the decision (valid/invalid) return the blockto the updated in the database.

Parameters

• block (dict) – Block to vote.

• previous_block_id (str) – The id of the previous block.

1.13. Developer Interface 19

Page 24: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

• decision (bool) – Whether the block is valid or invalid.

• invalid_reason (Optional[str]) – Reason the block is invalid

write_vote(block, vote, block_number)Write the vote to the database.

get_last_voted_block()Returns the last block that this node voted on.

get_unvoted_blocks()Return all the blocks that has not been voted by this node.

1.14 BigchainDB Consensus Plugins

BigchainDB has a pluggable block/transaction validation architecture. The default consensus rules can be extended orreplaced entirely.

1.14.1 Installing a plugin

Plugins can be installed via pip!

$ pip install bigchaindb-plugin-demo

Or using setuptools:

$ cd bigchaindb-plugin-demo/$ python setup.py install # (or develop)

To activate your plugin, you can either set the consensus_plugin field in your config file (usually~/.bigchaindb) or by setting the BIGCHAIN_CONSENSUS_PLUGIN environement variable to the name of yourplugin (see the section on Packaging a plugin for more about plugin names).

1.14.2 Plugin API

BigchainDB’s current plugin API exposes five functions in an AbstractConsensusRules class:

validate_transaction(bigchain, transaction)validate_block(bigchain, block)create_transaction(*args, **kwargs)sign_transaction(transaction, *args, **kwargs)verify_signature(transaction)

Together, these functions are sufficient for most customizations. For example:

• Replace the crypto-system with one your hardware can accelerate

• Re-implement an existing protocol

• Delegate validation to another application

• etc...

20 Chapter 1. Table of Contents

Page 25: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

1.14.3 Extending BigchainDB behavior

A default installation of BigchainDB will use the rules in the BaseConsensusRules class. If you only need tomodify this behavior slightly, you can inherit from that class and call super() in any methods you change, so longas the return values remain the same.

Here’s a quick example of a plugin that adds nonsense rules:

from bigchaindb.consensus import BaseConsensusRules

class SillyConsensusRules(BaseConsensusRules):

@staticmethoddef validate_transaction(bigchain, transaction):

transaction = super().validate_transaction(bigchain, transaction)# I only like transactions whose timestamps are even.if transaction['transaction']['timestamp'] % 2 != 0:

raise StandardError("Odd... very odd indeed.")return transaction

@staticmethoddef validate_block(bigchain, block):

block = super().validate_block(bigchain, block)# I don't trust Alice, I think she's shady.if block['block']['node_pubkey'] == '<ALICE_PUBKEY>':

raise StandardError("Alice is shady, everybody ignore her blocks!")return block

1.14.4 Packaging a plugin

BigchainDB uses setuptool’s entry_point system to provide the plugin functionality. Any custom plugin needs to addthis section to the setup() call in their setup.py:

entry_points={'bigchaindb.consensus': [

'PLUGIN_NAME=package.module:ConsensusRulesClass']

},

1.15 Monitoring

BigchainDB uses StatsD for monitoring. We require some additional infrastructure to take full advantage of its func-tionality:

• an agent to listen for metrics: Telegraf,

• a time-series database: InfluxDB, and

• a frontend to display analytics: Grafana.

We put each of those inside its own Docker container. The whole system is illustrated below.

1.15. Monitoring 21

Page 26: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

For ease of use, we’ve created a Docker Compose file (named docker-compose-monitor.yml) to define themonitoring system setup. To use it, just go to to the top bigchaindb directory and run:

$ docker-compose -f docker-compose-monitor.yml build$ docker-compose -f docker-compose-monitor.yml up

then point a browser tab to:

http://localhost:3000/dashboard/script/bigchaindb_dashboard.js

The login and password are admin by default. If BigchainDB is running and processing transactions, you should seeanalytics—if not, start BigchainDB and load some test transactions:

$ bigchaindb-benchmark load

then refresh the page after a few seconds.

22 Chapter 1. Table of Contents

Page 27: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

If you’re not interested in monitoring, don’t worry: BigchainDB will function just fine without any monitoring setup.

Feel free to modify the custom Grafana dashboard to your liking!

1.16 Licenses

Information about how the BigchainDB code and documentation are licensed can be found in the LICENSES.md file(in the root directory of the repository).

1.17 Contributing to the BigchainDB Project

If you’re interested in contributing to the BigchainDB project, you can find out how in the CONTRIBUTING.md file(in the root directory of the BigchainDB repository).

1.18 Release Notes

You can find a list of all BigchainDB releases and release notes on GitHub at:

https://github.com/bigchaindb/bigchaindb/releases

The CHANGELOG.md file contains much the same information, but it also has notes about what to expect in the nextrelease.

We also have a roadmap document in ROADMAP.md.

1.19 Glossary of Terms

BigchaindB client. A computer that can communicate with a BigchainDB cluster via the BigchainDB client-serverAPI.

BigchainDB cluster. A collection of servers running BigchainDB Server software and communicating with each othervia RethinkDB protocols. a BigchainDB cluster contains one logical RethinkDB datastore.

Block. A collection of up to 1000 transactions, plus other things. They get chained together by votes. See the sectionon the block model.

Federation. 1. An organization with members and some kind of governance structure. 2. The BigchainDB clusterowned and operated by a Federation’s members.

Federation node. A server running BigchainDB Server software, with permission to communicate with other nodesin a federation.

Node. See Federation node.

Transaction. The basic informational unit. A transaction can represent the creation or transfer of a digital asset. Seethe section on the transaction model.

Vote. Each federation node in a federation is required to vote on the validity of every block (i.e. whether the blockvalid or not). A node’s vote on a block also includes the id of the block it considers to be the previous block. See thesection on the vote model.

1.16. Licenses 23

Page 28: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

24 Chapter 1. Table of Contents

Page 29: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

CHAPTER 2

Indices and Tables

• genindex

• modindex

• search

25

Page 30: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

BigchainDB Documentation, Release 0.1.4

26 Chapter 2. Indices and Tables

Page 31: BigchainDB Documentation · CHAPTER 1 Table of Contents 1.1Introduction BigchainDB is a scalable blockchain database. You can read about its motivations, goals and high-level architecture

Index

Symbols__init__() (bigchaindb.core.Bigchain method), 17

BBigchain (class in bigchaindb), 17

Ccreate_block() (bigchaindb.Bigchain method), 19create_genesis_block() (bigchaindb.Bigchain method), 19create_transaction() (bigchaindb.Bigchain method), 17

Gget_last_voted_block() (bigchaindb.Bigchain method), 20get_owned_ids() (bigchaindb.Bigchain method), 18get_spent() (bigchaindb.Bigchain method), 18get_transaction() (bigchaindb.Bigchain method), 18get_tx_by_payload_hash() (bigchaindb.Bigchain

method), 18get_unvoted_blocks() (bigchaindb.Bigchain method), 20

Iis_valid_block() (bigchaindb.Bigchain method), 19is_valid_transaction() (bigchaindb.Bigchain method), 19

Ssign_transaction() (bigchaindb.Bigchain method), 17

Vvalidate_block() (bigchaindb.Bigchain method), 19validate_transaction() (bigchaindb.Bigchain method), 18verify_signature() (bigchaindb.Bigchain method), 17vote() (bigchaindb.Bigchain method), 19

Wwrite_block() (bigchaindb.Bigchain method), 19write_transaction() (bigchaindb.Bigchain method), 18write_vote() (bigchaindb.Bigchain method), 20

27