Blockchain Is Dead, Long Live Blockchain!Accountable State Machine Replication for Longlasting Blockchain

Alejandro Ranchal-PedrosaUniversity of Sydney

Sydney, Australiaalejandro.ranchalpedrosa@sydney.edu.au

Vincent GramoliUniversity of Sydney and EPFL

Sydney, Australiavincent.gramoli@sydney.edu.au

Abstract

The long-standing impossibility of reaching agreementrestricts the lifespan of blockchains. In fact, the consen-sus on a block to be appended to any blockchain succeedseither with some probability or at the condition that twothirds of the n replicas are not Byzantine. In the formercase, the probability that the blockchain fails grows expo-nentially with the number of newly appended blocks. Inthe latter case, the blockchain fails as soon as a coalitionbribes f = n/3 replicas. As a result, one may wonderwhether blockchains are doomed to fail.

In this paper, we answer this question in the negativeby proposing the first Longlasting Blockchain system,LLB. LLB builds upon the observation that blockchainsare rarely subject to benign faults. As opposed to prob-abilistic blockchains, LLB solves consensus determinis-tically when f < n/3. As opposed to Byzantine faulttolerant blockchains, it resolves a series of disagreementsby reducing eventually the number of deceitful replicasfrom n/3 ≤ f < 2n/3 to f ′ < n′/3 among a new setof n′ replicas. To demonstrate its effectiveness, we im-plement two coalition attacks and a zero loss payment ap-plication that forces replicas that misbehave to reimburseconflicting transactions. Finally, LLB outperforms theraw state machine replication at the heart of Facebook’sLibra and achieves performance comparable to a scalableblockchain that cannot tolerate n/3 failures.

1 Introduction

Blockchain systems promise to track ownership of assetswithout a central authority and thus rely heavily on solv-ing the consensus problem [45]. As consensus is knownto be impossible in the general model [27], blockchainsseem doomed to fail. Classic blockchains [45, 52, 41, 25]assume synchrony or that all messages are delivered in amaximum amount of time, and guarantee, despite tran-sient disagreements, that consensus will be reached withsome probability on the next block to be appended.Hence, the probability that such a blockchain fails growsexponentially fast with the number of blocks and thisprobability becomes 1 as the chain length tends to infin-ity [55]. The resulting disagreements can lead to doublespending as illustrated by the recent losses of $70, 000 inBitcoin Gold1 and $5.6 million in Ethereum Classic2.

More recent blockchains [9, 19, 29, 24] assume par-tial synchrony [22] and offer deterministic Byzantine faulttolerant consensus solutions but can only tolerate lessthan a third of malicious or Byzantine permissioned repli-cas [47]. By contrast with synchronous blockchains, it isnot sufficient to attack the network to double-spend. In-stead the attacker must form a coalition of a third of repli-cas to steal assets. But given the values of the assets thatcan be stolen, it is reasonable to assume that a third of thereplicas can get bribed after some time. Mitigation strate-gies require a regular arrival of new replicas to take over

1https://news.bitcoin.com/bitcoin-gold-51-attacked-network-loses-70000-in-double-spends/

2https://news.bitcoin.com/5-6-million-stolen-as-etc-team-finally-acknowledge-the-51-attack-on-network/

1

arX

iv:2

007.

1054

1v2

[cs

.DC

] 1

5 N

ov 2

020

the role of reaching consensus, so that the coalition neverexceeds a third of the current replicas. While this worksin theory, these strategies fail as soon as not enough hon-est newcomers can join. As a result, one may wonderwhether any blockchain is inherently doomed to fail in asufficiently long execution.

In this paper, we answer this question in the negative byproposing a blockchain with deterministic guarantees thattolerate a majority of failures. More precisely, our sys-tem assumes partial synchrony [22], that message deliverytime has an unknown upper bound, to tolerate d2n/3e−1deceitful replicas and dn/3e − 1 benign replicas. LLBcomprises an accountable state machine replication thatignores messages that are not properly signed and jus-tified and ensures the following: If deceitful faults leadhonest replicas to disagree, then these honest replicas pro-duce an undeniable proof-of-fraud to rightfully excludeat least bn/3c + 1 deceitful replicas, and include newreplicas, while resolving the disputed disagreement. Tothis end, we build upon recent theoretical accountabilityresults [13, 14] to extend PeerReview [31] that suspectsfailures forever in a partially synchronous model intoLLB that exclude undeniably identified deceitful replicas.After a finite number of these exclusions, LLB guaran-tees that the remaining n′ replicas contain less than n′/3Byzantine faults, which leads to consensus.

Note that the deceitful failure model differs radicallyfrom the failures assumed in traditional distributed sys-tems and is interesting in its own right. As opposedto datacenters [16], cloud services [40] or distributeddatabases [32] where most faults are omissions and rarecommissions are due to unlucky events (e.g., disk er-rors [16]), in blockchain payment systems deceitful faultsare more frequent than benign faults. Indeed, blockchainsare more subject to well-fomented attacks to steal as-sets [35, 26] but, when not deceitful, blockchain repli-cas are more likely up and running than crashed: they arecarefully monitored to generate financial rewards to theirowner [45, 52]. We thus distinguish two new classes offaults: (i) deceitful faults that could directly affect safety(e.g., sending correctly formatted messages with mali-cious content to try to influence the execution of honestreplicas) and (ii) benign faults that do not risk to violatesafety (e.g., omission faults, unintelligible messages orout-of-date well-formatted messages).

To demonstrate the applicability of LLB, we build a

zero loss payment application with Bitcoin transactionsand illustrate its performance under two distinct coalitionattacks. The key to zero loss is that we consider fun-gible assets and request replicas to deposit sufficient as-sets before participating in the consensus. If two conflict-ing transactions are found after the exclusion of the de-ceitful replicas responsible for a transient disagreement,then the zero loss payment system executes the first ofthese conflicting transactions and withdraws the depositedamount of the deceitful replicas to reimburse the secondtransaction. As opposed to classic blockchain paymentsystems [45, 52, 41, 25] where recovering from forks aposteriori is the norm, our LLB guarantees deterministicagreement—no forks—in good executions but recoversonly in unlucky cases. As opposed to more recent Byzan-tine fault tolerant blockchains [9, 28, 19, 29, 24], LLB re-covers eventually from a state with q < min(n/3, n− f)benign replicas and f − q < 2n/3 deceitful replicas.Our results show that the impact of the attacks decreasesrapidly with the system size.

LLB presents some limitations. First, to hold nodesaccountable for their actions, we need an expensive cryp-tographic primitive that adds up to the CPU demand ofblockchain systems and cannot be minimized with thresh-old encryption or message authentication codes. Despitethis, we show that by deciding multiple proposals, LLBscales to 90 simple 4 vCPU machines (§5) to outper-form the HotStuff raw state machine replication [42] at theheart of the Libra blockchain [24] by 5.6 times. Second,LLB cannot guarantee termination when 2n/3 ≤ f ≤ n,hence even though no assets are lost, the system risks tostop being available. Third, the zero loss payment sys-tem does not imply full satisfaction of honest users: Con-sider that a user thinks they succeed in buying a rare itemsold on a first-come first-served basis, say Guernica by Pi-casso, they might be disappointed when they realize (eventhough they were not charged) that due to a transient forksomeone else acquired it.

We present the background (§2), our problem (§3), ourLLB solution (§4), our evaluation (§5) and the paymentsystem application (§6) before presenting the related work(§7) and concluding (§8).

2

2 Background and Preliminaries

A blockchain system [45] is a distributed system main-taining a sequence of blocks that contains valid (crypto-graphically signed) transactions indicating how assets areexchanged between accounts.

Accountability. The replicas of a blockchain systemare, by default, not accountable in that their faults often goundetected. For example, when a replica creates a fork, itmanages to double spend after one of the branches whereit spent coins vanishes. This prevents other replicas fromdetecting this misbehavior in order to hold this replica ac-countable for its actions. Recently, Polygraph [13, 14] in-troduced accountable consensus (Def. 1) as the problemof solving consensus if f < n/3 and detecting eventuallyin case of disagreement fd ≥ n/3 faulty replicas.

Definition 1 (Accountable Consensus). The problem ofaccountable consensus is to solve consensus when thenumber of Byzantine faults is f < n/3 and for every hon-est replica to eventually output at least fd = dn/3e faultyreplicas if two honest replicas output distinct decisions.

Byzantine state machine replication. A ByzantineState Machine Replication (SMR) [12, 33] is a replicatedservice that accepts deterministic commands from clientsand totally orders these commands using a consensus pro-tocol so that upon execution of these commands, everyhonest replica ends up with the same state despite Byzan-tine or malicious replicas. The instances of the consensusexecute in sequence, one after the other, starting from in-dex 0. We refer to the consensus instance at index i asΓi.

Traditionally, provided honest replicas propose a value,the Byzantine consensus problem [47] is for every honestreplica to eventually decide a value (consensus termina-tion), for no two honest replicas to decide different values(agreement) and for the decided value to be one of the pro-posed values (validity). In this paper, we consider how-ever a variant of the Byzantine consensus (Def. 2) usefulfor blockchains [43, 21, 19] where the validity requiresthe decided value to be a subset of the union of the pro-posed values.

Definition 2 (Set Byzantine Consensus). Assuming thateach honest replica proposes a proposal, the Set Byzan-tine Consensus (SBC) problem is for each of them to de-cide on a set in such a way that the following propertiesare satisfied:

• SBC-Termination: every honest replica eventuallydecides a set of transactions;

• SBC-Agreement: no two honest replicas decide ondifferent sets of transactions;

• SBC-Validity: a decided set of transactions is a non-conflicting set of valid transactions taken from theunion of the proposed sets; and if all replicas arehonest and propose a common valid non-conflictingset of transactions, then this set is the decided set.

SBC-Validity includes two predicates, the first statesthat transactions proposed by Byzantine proposers couldbe decided as long as they are valid and non-conflicting(i.e., they do not withdraw more assets from one accountthan its balance); the second one is necessary to preventany trivial algorithm that decides a pre-determined valuefrom solving the problem. As a result, we consider thata consensus instance Γi outputs a set of enumerable deci-sions out(Γi) = di, |di| ∈ N that all n replicas replicate.We refer to the state of the SMR at the i-th consensus in-stance Γi as all decisions of all consensus instances up tothe i-th consensus instance.

Solving the Set Byzantine Consensus (SBC) A clas-sic reduction [5, 6, 18, 13] of the problem of multi-valueconsensus, that accepts any ordered set of input values,to the problem of binary consensus, that accepts binaryinput values, proved promising to solve the SBC prob-lem (Def. 2) when f < n/3 [19]. The idea consists ofexecuting an all-to-all reliable broadcast [8] to exchangen proposals: any delivered proposal is stored in an arrayproposals at the index corresponding to the identifier ofthe broadcaster. A binary consensus at index k is startedwith input value 1 for each index k where a proposal hasbeen recorded. Once n−f proposals are delivered locally,a binary consensus at the remaining indices 0 ≤ ` < nwhere ` 6= k is started with input value 0. The resultsof these concurrent binary consensus instances is storedinto a bitmask array. And applying the bitmask to the

3

benignfaults

omissionfaults

commissionfaultsdeceitful

faults

fault in closed networks fault in blockchain networks

no faults (honest)

Byzantine fault

severity

Figure 1: Unlike closed networks, blockchain networksexperience more likely deceitful faults, due to well-fomented attacks, than benign faults (which could eitherbe omission or some commission faults) that are harmless.

proposal array yields a sequence of proposals whose con-tent is the output of consensus. Polygraph is the account-able variant where replicas broadcast certificates, sets of2n/3 messages signed by distinct replicas, each time theyreliably broadcast or decide a binary value.

3 The Longlasting BlockchainProblem

Considering the classic distributed system model [12, 33]where messages are delivered within bounded but un-known time (i.e., partial synchrony [22]) and failures areByzantine, solving the longlasting blockchain problemis to solve consensus when possible (f < n/3), andto recover from a situation where consensus is violated(n/3 ≤ f < 2n/3) to a situation where this violation isresolved (with f ′ < n′/3).

Attacking the SBC solution In the SBC solution pre-sented above, deceitful replicas can form a coalition off ≥ n/3 replicas to lead honest replicas to a disagreementby equivocating (sending distinct messages) to differentpartitions of honest replicas, with one of two coalition at-tacks:

1. Reliable broadcast attack: deceitful replicas mis-behave during the reliable broadcast by sending dif-ferent proposals to different partitions, leading hon-est replicas to end up with distinct proposals at thesame index k.

2. Binary consensus attack: deceitful replicas vote foreach binary value in each of two partitions for thesame binary consensus leading honest replicas to end

up with different bits in the same index of their bit-mask.

Note that deceitful replicas do not benefit from combiningthese attacks: If two honest replicas deliver different pro-posals at index k, the disagreement comes from them out-putting 1 at the corresponding binary consensus instance.Similarly, forcing two honest replicas to disagree duringthe kth binary consensus only makes sense if they bothhave the same corresponding proposal at index k.

The deceitful Byzantine failure model. We introducethe deceitful failure model that is a refinement of theByzantine failure model [36], radically different from theclosed network failure models [16, 32, 40] where mostByzantine failures are omissions and very few are com-missions (cf. Fig. 1). This contrast stems from the obser-vation that blockchain payment systems incentivize repli-cas to either fail by attacking the network or minimizedowntime to maximize profit [45, 52]. A deceitful faultconsists of sending a message that violates the protocolto deceive honest replicas and try to reach a disagreementwhereas a benign fault consists of a non-deceitful Byzan-tine fault (e.g., sending a stale message). We refer to areplica that commits a deceitful (resp. benign) fault asa deceitful replica (resp. benign replica) and fail withouttrying to cause any harm to the system. We also denote thedeceitful ratio (f−q)/n by δ. A replica that is not Byzan-tine is honest. Let n be the initial number of replicas in oursystem, we thus assume a maximum of f − q < 2n/3 de-ceitful replicas and q < min(n/3, n− f) benign replicas.The adversary that controls these Byzantine replicas is dy-namic in that f can change over time, however, we as-sume that the adversary experiences static periods duringwhich each consensus instance fully executes and the hon-est replicas do not become faulty, and there exists a poolof replicas among which at least 2n/3 are honest replicasand the rest are deceitful. Like in classic blockchain mod-els, we assume the presence of unforgeable signatures.

Longlasting Blockchain. A Longlasting Blockchain(LLB) is a Byzantine SMR that allows for some consen-sus instances to reach a disagreement before fixing thedisagreement by merging the branches of the resultingfork and deciding the union of all the past decisions us-ing SBC (Def. 2). More formally, an SMR is an LLB if

4

it ensures termination, agreement and convergence as de-fined below:

Definition 3 (Longlasting Blockchain Problem). An SMRis an LLB if the following properties are all satisfied:

1. Termination: For all k > 0, consensus instance Γkterminates, either with agreement or disagreement.

2. Agreement: If f < n/3 during Γk, then honestreplicas executing Γk reach agreement.

3. Convergence: If f − q < 2n/3 and q <min(n/3, n− f), then in any sufficiently long staticperiod of the adversary there is a finite number ofdisagreements after which honest replicas agree.

Termination does not necessarily imply agreementamong honest replicas whereas agreement is the classicproperty of the consensus problem. Convergence pre-serves the assets of honest replicas by guaranteeing thatthere is a limited number of disagreements (this numberis 0 if f − q < n/3) after which agreement is maintained,within a sufficiently long static period.

4 Description of the LonglastingBlockchain

In this section we detail our system. Its two main ideas areto replace deceitful replicas undeniably responsible for afork by new replicas to converge towards a state whereconsensus can be reached, and to fund conflicting transac-tions that were wrongly decided. We will show that LLBsolves the longlasting blockchain problem. As depicted inFigure 2, we present below the components of our system,namely the ASMR (§4.1) and the BM (§4.2) but we deferthe description of the zero loss payment system (§6).

As long as new requests are submitted by a client toa replica, the payment system component of the replicaconverts it into a payment that is passed to the BM com-ponent. As depicted in Figure 2, when sufficiently manypayments are present, the BM issues a batch of requests tothe ASMR that, in turn, proposes it to the consensus com-ponent, which exchanges messages though the networkfor agreement among honest replicas to be reached. If adisagreement is detected, then the account of the deceit-ful replica is slashed. Consider that Alice (A) attempts to

Block Manager (BM)

Accountable SMR(ASMR)

Zero-loss Payment System

replica pi

Consensus

t=A B

b=tx-batch

propose(b)

decision(b,b’…)

refund(B)

Block Manager (BM)

Accountable SMR(ASMR)

Zero-loss Payment System

replica pj

b’=tx-batch

reportfraud(k)

refund(B)

exclude(pk)

propose(b’)

decision(b,b’…)

$1M

Network

Consensussend(*) recv(*) send(*) recv(*)

reportfraud(k)

exclude(pk)

t’=A C$1M1

2

3

4

5

6

7

7 2

1

3

4

5

6 7

7

include(*)include(*)

Figure 2: The distributed architecture of our LLB systemrelies on ASMR, BM and the payment system deployedon several replicas. Ë Each replica batches some pay-ment requests illustrated with Ê a transfer t (resp. t′) of$1M from Alice’s account (A) to Bob’s (B) (resp. Carol’s(C)). Consider that Alice has $1M initially and attemptsto double spend by modifying the code of a replica pkunder her control so as to execute the coalition attack. Ì–Î The ASMR component detects the deceitful replica pkthat tried to double spend, the associated transactions tand t′ and account A with insufficient funds. It choosestransaction t and discards t′, Ï notifies BM that Ð ex-cludes or replaces replica pk and Ð refunds B with pk’sdeposit.

double spend by spending her $1M with both Bob (B) andCarol (C) in t and t′, respectively, and hacking the codeof replica pk that commits deceitful faults for disagree-ment to occur. Once the ASMR detects the disagreement,BM is notified, which results in excluding the replica pkfrom the ASMR (potentially replacing it by newly joinedreplicas) and funding B with pk’s deposit.

4.1 Accountable SMR (ASMR)We present our ASMR that consists of running an infi-nite sequence of five consecutive actions: À the account-able consensus (Def. 1) that tries to decide upon a newset of transactions, Á a confirmation that aims at confirm-ing that the agreement was reached, –à a membershipchange that aims at replacing deceitful replicas responsi-

5

Instance 𝛤kASMR consensus confirmation exclude include

disagreementfound

upondecision

agreement onpofs

uponrequest

add newreplicas

decisionscombined

uponrequest

upondecision

reconcile2 3 4 5 ASMR consensusoptionaltime Instance 𝛤k+1

1 1

Figure 3: If there are enqueued requests that wait to be served, then a replica starts a new instance Γk by participatingin an ASMR consensus phase À; a series of optional phases may follow: Á the replica tries to confirm this decisionto make sure no other honest replica disagrees, Â it invokes an exclusion consensus if it finds enough proofs-of-fraud(PoFs), Ã it then potentially includes new replicas to compensate for the exclusion, and Ä merges the two batchesof decided transactions. Some of these phases complete upon consensus termination (in black) whereas other phasesterminate upon simple notification reception (in grey). The replica starts a new instance Γk+1 if there are otherenqueued requests to be served, hence participating in a new ASMR consensus phase À that may succeed, in whichcase none of the optional phases immediately follow.

ble for a disagreement by new replicas and Ä a reconcili-ation phase that combines all the disagreement decisions,as depicted in Figure 3.

The phases of ASMR For each index, ASMR first runsthe accountable consensus (§2) to try to agree on a setof transactions then it optionally runs the four subsequentphases to recover from the possible disagreement.

À ASMR consensus: Honest replicas propose a set oftransactions, which they received from clients, to theaccountable consensus (Def. 1) in the hope to reachagreement. When the consensus terminates, all hon-est replicas agree on the same decision or honestreplicas disagree: they decide distinct sets of trans-actions.

Á Confirmation: As honest replicas could be un-aware of the other decisions, they enter a confirma-tion phase waiting for messages coming from moredistinct replicas than what consensus requires. Ifthe confirmation leads honest replicas to detect dis-agreements, i.e., they receive certificates supportingdistinct decisions, then they start the membershipchange. This phase may not terminate as an hon-est replica needs to deliver messages from more than(δ+1/3) ·n replicas (due to the number of ‘conflict-ing histories’ [48]) to guarantee that no disagreementwas possible by a deceitful ratio δ, however, it doesnot prevent Γk from terminating as they proceed inparallel.

 Membership change (exclusion): Honest replicasmay identify deceitful replicas by cross-checking theset of signed messages or certificates. Once a dis-agreement is detected, the certificates are used asproofs-of-fraud (PoFs). These PoFs cannot be falsi-fied as they indicate conflicting signed messages thatthe same replica should not have both sent if it washonest. If honest replicas detect fd = dn/3e de-ceitful replicas, they start an exclusion consensus toagree on PoFs. Note that they could instead start assoon as they detect one deceitful replica, however,waiting for at least fd guarantees that a membershipchange is necessary and will help remove many de-ceitful replicas from the same coalition at once. Asthese membership consensuses are also accountable,they may trigger another confirmation or another ex-clusion consensus, hence the arrow from à to Á onFigure 3.

à Membership change (inclusion): To compensatewith the excluded replicas, an inclusion consensusadds new candidate replicas (among the pool of can-didates). This inclusion consensus waits to deliverat least dn′/2e proposals, each containing n − n′

new replicas to include where n′ is the number ofreplicas at the start of the inclusion consensus (af-ter the exclusion consensus). Once consensus fin-ishes, replicas deterministically choose n−n′

dn′/2e repli-cas from each proposal, to guarantee a fair distribu-tion of included replicas across all honest replicas.To prevent the deceitful ratio from increasing after

6

an inclusion consensus, we rely on a deterministicfunction that, given a disagreement upon replicas toinclude, chooses some of them instead to maintainthe total number n of replicas.

Ä Reconciliation: Once the membership change fin-ishes, the reconciliation starts by combining alltransactions that were decided by distinct honestreplicas in the preceding disagreement. These trans-actions are ordered through a deterministic function,whose simple example is a lexicographical order butcan be made fair by rotating over the indices of theinstances.

Once the current instance Γk terminates, another in-stance Γk+1 can start, even if it runs concurrently witha confirmation or a reconciliation at index k.

Algorithm 1 Membership change at replica pi1: State:2: Γk , kth instance of ASMR consensus pi participates to with field:3: proc ⊂ I , the set of participating replicas not proved deceitful yet4: pofs , the set of proofs-of-fraud (PoFs), initially ∅5: cons-exclude, the set of PoFs output by consensus, initially ∅6: cons-include, the set of new replicas output by consensus, initially ∅7: inc-prop, the set of new replicas that replica pi proposes to add8: deceitful ∈ I , the identity of an agreed deceitful replica, initially ∅9: fd, the threshold of proofs-of-fraud to recover, dn/3e by default

10: Upon receiving a list of proofs-of-fraud pofs:11: if (verify( pofs)) then B if PoFs are correctly signed12: pofs.add( pofs) B add PoFs on distinct replicas13: if (size(pofs) ≥ fd) then B enough to change members14: if pending Γk then v ← Γk .stop() B it may violate agreement15: cons-exclude ← Γk.ex-propose(pofs) B exclusion consensus16: for all pof in cons-exclude do B for all decided PoFs17: deceitful ← pof .get-deceitful() B get deceitful18: detected-fraud(deceitful) B application punishment19: pofs ← pofs \ pof B discard the treated pofs20: Γk.proc ← Γk.proc \ deceitful B exclude deceitful21: cons-include ← Γk.inc-propose(inc-prop) B inclusion cons.22: for all new-replica in cons-include do B for all new to inc.23: set-up-connection(new-replica) B new replica joins24: send-catchup(new-replica) B get latest state25: Γk.proc ← Γk.proc ∪new-replicaB include new replica26: if Γk is stopped then goto À B restart cons.

Membership change details. The membership change(Alg. 1), proposes PoFs pofs to an exclusion consensusex-propose to exclude deceitful replicas and runs an in-clusion consensus inc-propose to potentially add newlyjoined replicas. To this end, replica pi maintains the cur-rent consensus instance Γk, the deceitful replicas amongthe whole set proc of replica ids, new candidate replicas

inc-prop, a threshold fd of detected deceitful replicas,two sets cons-exclude of decided PoFs and cons-includeof decided new replicas. Each PoF is valid if it con-tains equivocating messages signed by the same replica(line 11). The exclusion consensus starts once there aremore PoFs than fd (line 13) but as opposed to waitingfor n − f signed responses, it only waits for dn

′

2 e signedresponses from distinct replicas where n′ = n − fd aseven during disagreement, PoFs validity can be checkedlocally. It may however be necessary to stop a pendingconsensus (line 14) before restarting it with the new setof replicas (line 26). At the end, the excluded replicas arepunished by the application layer (line 18) and the newreplicas are included (lines 21 to 25) to compensate theloss. Once the inclusion consensus finishes, the honestreplicas share their state with the newly added replicas(line 24). Typically, new replicas simply verify the re-ceived certificates and PoFs.

Remarks. Honest replicas should still accept certifi-cates with signatures from excluded replicas in consensusinstances that follow the membership change, or from in-cluded replicas whose inclusion was finally reverted, inthe event of a disagreement. This transient acceptance al-lows these honest replicas to catch up on the new set ofmembers without reverting any decision. We separate in-clusion and exclusion in two consensus instances to avoiddeciding to exclude and include the replicas proposed bythe same replica.

4.2 Blockchain Manager (BM)We now present the Blockchain Manager (BM) that buildsupon ASMR to merge the blocks in multiple brancheswhen forks are detected. Once a fork is identified, theconflicting blocks are not discarded as it would be thecase in classic blockchains when a double spending oc-curs, but they are merged. Upon merging blocks, BM alsocopes with conflicting transactions, as the ones of a pay-ment system, by keeping one transaction and reimburs-ing its conflicting ones. We defer to §6 the details of theamount replicas must have on a deposit to guarantee thisreimbursement.

Similarly to Bitcoin [45], BM accepts transaction re-quests from a permissionless set of users. In particu-lar, this allows users to use different devices or wallets

7

n/3

single branch

1 outdated branch

multiple outdated branches

0 ≤ 𝛿’ < 1/3

1/3 ≤ 𝛿’ < 1/2

1/2 ≤ 𝛿 < 5/9

5/9 ≤ 𝛿 < 2/3

1/2 ≤ 𝛿 < 5/9

1/3 ≤ 𝛿 < 1/2

two updated branches

1/2 ≤ 𝛿’ < 5/9

three updated branches

5/9 ≤ 𝛿 < 2/3

1

2

3

Figure 4: After a finite number of membership changesduring a static period of the adversary, the deceitful ratiodecreases from the worst case δ < 2/3 Ì to the agreementcase where δ < 1/3 Ê. This is guaranteed even though thedeceitful ratio passes through intermediate values, for ex-ample δ < 5/9 and δ < 1/2 Ë. In particular, each mem-bership change guarantees that the deceitful ratio neverincreases and eventually decreases.

to issue distinct transactions withdrawing from the sameaccount—a feature that is not offered in payment systemswithout consensus [17]. In contrast with Bitcoin, our sys-tem does not incentivize all users to take part in tryingto decide upon every block, instead a restricted set ofpermissioned replicas have this responsibility for a givenblock. This is why BM, combined with ASMR, offers anopen permissioned blockchain.

Guaranteeing consistency across replicas. Buildingupon the accountability of the underlying ASMR that re-solves disagreement, BM features a block merge to re-solve forks by excluding deceitful replicas and includ-ing new replicas. As depicted in Figure 4, a consensusmay reach a disagreement if f ≥ n/3, resulting in thecreation of multiple branches or blockchain forks. BMbuilds upon the membership change of ASMR in orderto recover from forks. In particular, the fact that ASMRexcludes fd deceitful replicas each time a disagreement

occurs guarantees that the ratio of deceitful replicas δ =(f − q)/n converges to a state where consensus is guar-anteed (Lemma 4.1). The maximum number of branchesthat can result from forks depends on the number q of be-nign faults and the number f−q of deceitful faults as wasalready shown for histories of SMRs [48].

Updated/outdated replicas and branches. Figure 4depicts an example with updated and outdated branches,as described below. After the merge of some branchesterminates, some replicas can become aware of the re-sulting branch—we call them updated replicas and wecall the resulting branch the updated branch. Other repli-cas may remain unaware of the resulting branch—we callthem outdated replicas. As a result, outdated replicas maywrongly believe that an old branch still exists—we callsuch a branch an outdated branch.

Fork resolution. Figure 4 also depicts a series of mem-bership changes leading evntually to a state in which anagreement can be reached (δ < n/3). Ì If 5/9 ≤ δ <2/3, then deceitful replicas can create as many as n − fbranches (i.e., the number of honest replicas) but againreplicas will learn about the updated branches. After afinite number of membership changes, the deceitful ratiostrictly decreases, for example to 1/2 ≤ δ′ < 5/9. Atthis point the number of branches is at most 3 updatedbranches. Ë From 1/2 ≤ δ < 5/9, the process continuesand a finite number of membership changes can lead tosay 1/3 ≤ δ′ < 1/2. At this point the number of branchesis at most 2 updated branches. Ê From 1/3 ≤ δ < 1/2,the process continues and a finite number of membershipchanges leads for example to 0 ≤ δ′ < 1/3. At thispoint the number of branches is at most 1 updated branch.Hence the system eventually recovers a state where con-sensus is guaranteed until the static period ends as weshow in Theorem 4.2, at which point new deceitful repli-cas may lead the system back to Ì.

In memory transactions. LLB is a blockchain thatinherits the same Unspent Transaction Output (UTXO)model as Bitcoin [45]; the balance of each account in thesystem is stored in the form of a UTXO table. In con-trast with Bitcoin, the number of maintained UTXOs iskept to a minimum in order to allow in-memory optimiza-tions. Each entry in this table is a UTXO that indicates

8

Algorithm 2 Block merge at replica pi1: State:2: Ω, a blockchain record with fields:3: deposit , an integer, initially 04: inputs-deposit , a set of deposit inputs, initially in the first deposit5: punished-acts , a set of punished account addresses, initially ∅6: txs , a set of UTXO transaction records, initially in the genesis block7: utxos , a list of unspent outputs, initially in the genesis block

8: Upon receiving conflicting block block : B merge block9: for tx in block do B go through all txs

10: if (tx not in Ω.txs) then B check inclusion11: CommitTxMerge(tx) B merge tx, go to line 1712: for out in tx .outputs do B go through all outputs13: if (out.account in Ω.punished-acts) then B if punished14: PunishAccount(out.account) B punish also this new output15: RefundInputs() B refill deposit, go to line 2416: StoreBlock(block) B write block in blockchain

17: CommitTxMerge(tx):18: toFund ← 019: for input in tx .inputs do B go through all inputs20: if (input not in Ω.utxos) then B not spendable, need to use deposit21: Ω.inputs-deposit.add(input) B use deposit to refund22: Ω.deposit ← Ω.deposit − input.value B deposit decreases

in value23: else Ω.consumeUTXO(input) B spendable, normal case

24: RefundInputs():25: for input in Ω.inputs-deposit do B go through inputs that used

deposit26: if (input in Ω.utxos) then B if they are now spendable27: Ω.consumeUTXO(input) B consume them28: Ω.deposit ← Ω.deposit + input.value B and refill deposit

some amount of coins that a particular account, the ‘out-put’ has. When a transaction transferring from source ac-counts s1, ..., sx to recipient accounts r1, ..., ry executes,it checks the UTXOs of accounts s1, ..., sx. If the UTXOamounts for these accounts are sufficient, then this execu-tion consumes as many UTXOs as possible and producesanother series of UTXOs now outputting the transferredamounts to r1, ..., ry as well as what is potentially left tothe source accounts s1, ..., sx. Maximizing the numberof UTXOs to consume helps keeping the table compact.Each replicas can thus generally access the UTXO tabledirectly in memory for faster execution of transactions.

Protocol to merge blocks. As depicted in Alg. 2,the state of the blockchain Ω consists of a set of in-puts inputs-deposit (line 4), a set of account addressespunished-acts (line 5) that have been used by deceitfulreplicas, a deposit (line 3), that is used by the proto-col, a set txs of transactions and a list utxos of UTXOs.The algorithm relies on a usual propagation of blocks

by broadcasting on the network and starts upon recep-tion of a block that conflicts with an existing knownblock of the blockhain Ω by trying to merge all trans-actions of the received block with the transactions of theblockchain Ω (line 11). This is done by invoking the func-tion CommitTxMerge (lines 17–23) where the inputs getappended to the UTXO table and conflicting inputs arefunded with the deposit (line 22) of a deceitful replica.We explain in §6 how to build a payment system with asufficient deposit to remedy double spending attempts.

Cryptographic techniques. To provide authenticationand integrity, the transactions are signed using the El-liptic Curves Digital Signature Algorithm (ECDSA) withparameters secp256k1, as in Bitcoin [45]. Each hon-est replica assigns a strictly monotonically increasing se-quence number to its transactions. The network com-munications use gRPC between clients and replicas andraw TCP sockets between replicas, but all communica-tion channels are encrypted through SSL. Finally, the ex-cluding protocol (Alg. 1) also makes use of ECDSA forPoFs or authentifying the sender of messages responsiblefor disagreement. One may think that message authen-tication codes (MACs) or threshold encryption could bemore efficient alternatives to this classic public-key cryp-tosystem, however, threshold encryption cannot be usedto trace back the faulty users as they are encoded in lessbits than what is needed to differentiate users, and MACsare insufficient to provide this transferrable authentica-tion [15].

4.3 Fault tolerance and intuition of theproof

In this section, we provide the intuition of the proof thatLLB solves the Longlasting Blockchain problem (Def. 3)hence recovering from of a majority of failures. To thisend, we show that LLB solves termination, agreementwhen f < n/3 and convergence when f − q < 2n/3and q < min(n/3, n− f).

A membership change is called successful if its seriesof inclusion and exclusion consensus ends up with allhonest replicas deciding the same set of replicas.

Lemma 4.1. In LLB if the number of deceitful replicasis f − q < 2n/3 and the number of benign replicas is

9

q < n − f then any disagreement on Γ is followed by amembership change that terminates successfully, and thatremoves at least fd ≥ n/3 replicas.

Proof. The proof is in two parts, termination and success,but we need first to show that a membership change fol-lows any disagreement on Γ. Note that f − q < 2n

3 im-plies deceitful replicas cannot decide on their own. Sinceeach honest replica decides on a single decision, there canonly be as many (finite) decisions as honest replicas. Thenumber of branches produced by Γ is thus upper-boundedby the number n − f of the number of honest replicas.Since honest replicas exchange certificates as in Poly-graph [13, 14], they identify at least fd ≥ n/3 deceitfulreplicas responsible for a disagreement.

1. Termination. The existence of a disagreement im-plies q < n/3. As deceitful replicas help termi-nating by definition, consider the most difficult sce-nario where all failures left are benign. The exclu-sion consensus thus excludes all deceitful replicasleading to n′ = n − (f − q). Since both inclusionand exclusion consensus instances wait for at leastn′

2 = n−(f−q)2 messages from distinct replicas, it

follows that we need n′

2 > q for them to terminate.Thus, n−(f−q)

2 > q ⇐⇒ q < n − f , which isalways true if f < 2n/3 and q < n/3.

2. Success. Now that we know that the membershipchange completes, we show that it succeeds. Byconstruction, honest replicas only start the member-ship change if they gather at least fd PoFs from dis-tinct replicas, which later they propose during theexclusion consensus. As a result, since the exclu-sion consensus decides on at least one correct pro-posal, at least fd nodes are removed. If there is adisagreement during this exclusion consensus, theunion of both PoFs is eventually adopted as the de-cision, which excludes even more deceitful replicas,hence leading to success.

Recall that by assumption (§3) there exists a pool with2n/3 honest replicas from which honest replicas select asubset to propose to the inclusion consensus. For sim-plicity, we also assume that no replica from this pool isincluded twice.

Theorem 4.2 (Convergence). In LLB if the number of de-ceitful replicas is f−q < 2n/3 and the number of benignreplicas is q < min(n/3, n − f), then in any sufficientlylong static period of the adversary there is a finite numberof disagreements after which honest replicas agree.

Proof. By Lemma 4.1, we know that every disagreementleads to a membership change in which fd ≥ n/3 will beremoved. Note first that the existence of a decision im-plies q < n/3. The inclusion consensus does not increasethe deceitful ratio, since the inclusion consensus does notinclude more replicas than the number of excluded repli-cas by the exclusion consensus and all excluded replicasare deceitful. As the inclusion consensus decides at leastn′/2 proposals where n′ = n− fd and the remaining de-ceitful replicas are f ′ = f − q − fd < n/3 < n′/2, itfollows that at least one proposal from one honest replicawill be decided. As the pool of joining candidates is fi-nite and no replica is included more than once, it followsthat the deceitful ratio will eventually decrease in any suf-ficiently long static period of the adversary. Some inclu-sion consensus will thus eventually lead to a deceitful ra-tio δ < 1/3 and agreement is eventually reached fromthen on, since q < n/3.

Agreement and termination follow from the fact thatthe consensus algorithm at the heart of LLB terminateswhen f < n/3 as was previously shown [18, 49].

5 Blockchain Experimental Evalua-tion

This section is dedicated to answer the following ques-tions: Does LLB offer practical performance in a geo-distributed environment? When f < n/3 how doesASMR perform compared to the raw state machine repli-cation at the heart of Facebook Libra? When f ≥ n/3,what is the impact of large scale coalition attacks on therecovery of ASMR? We defer the evaluation of a zero-losspayment application to §6.

Experimental settings. To evaluate LLB, we compareits performance to (i) HotStuff [54], the state machinereplication with linear message complexity a variant ofwhich has been integrated in Facebook’s Libra [24];(ii) Red Belly Blockchain [19] whose performance scales

10

10 20 30 40 50 60 70 80 90Number of replicas

0

10

20

30LLB Polygraph HotStuff Red Belly

Thro

ughp

ut

(103 t

x/s)

Figure 5: Throughput of LLB (decisions and confirma-tions) compared to that of Polygraph [13], HotStuff [54]and the Red Belly Blockchain [19].

to hundreds of nodes, and (iii) Polygraph [13] an ac-countable blockchain prototype that does neither excludenor include replicas, and whose verification technique isnot accountable. For all three, we deployed the originalcode from the corresponding authors [54, 19, 13]. in twodistributed settings of c4.xlarge Amazon Web Services(AWS) instances equipped with 4 vCPU and 7.5 GiB ofmemory: (i) a LAN with up to 100 machines and (ii) aWAN with up to 90 machines. We evaluate LLB with anumber of failures f up to d 2n

3 e − 1, however, when notspecified we fix f − q = d5n/9e − 1 and q = 0. Allerror bars represent the 95% confidence intervals and theplotted values are averaged over 3 to 5 runs. All trans-actions are ˜400-byte Bitcoin transactions with ECDSAsignatures [45].

5.1 LLB vs. HotStuff, Red Belly and Poly-graph

Figure 5 compares the performance of LLB, Red BellyBlockchain and Polygraph deployed over 5 availabilityzones of 2 continents California, Oregon, Ohio, Frankfurtand Ireland (exactly like Polygraph experiments [13]).For LLB, we only represent the decision throughput thatreaches 16, 626 tx/sec at n = 90 as the confirmationthroughput is similar (16, 492 tx/sec). As only LLB toler-ates f ≥ n/3, we fix f = 0.

First, Red Belly Blockchain offers the highest through-put. As expected it outperforms LLB due to its lack ofaccountability: it does not require messages to piggybackcertificates to detect PoFs. Both solutions solve SBC sothat they decide more transactions as the number of pro-posals enlarges and use the same batch size of 10, 000transactions per proposal. As a result ASMR scales prettywell: the cost of tolerating f ≥ n/3 failures even appearsnegligible at 90 replicas.

0369

1215 1000ms

500ms200ms

aws-likegamma

10 20 30 40 50 60 70 80 90 100Number of replicas

01224364860 200.0ms, rbbcast

500.0ms, rbbcast1000.0ms, rbbcastDi

sagr

eem

ents

Figure 6: Disagreeing decisions per number of replicasfor a variety of uniform delays and for delays generatedfrom a Gamma distribution and a distribution that drawsfrom observed AWS latencies when equivocating whilevoting for a decision (top), and when equivocating whilebroadcasting the proposals (bottom), for f = d5n/9e − 1and q = 0.

Second, HotStuff offers the lowest throughput even if itdoes not verify transactions. Note that HotStuff is bench-marked with its dedicated clients in their default config-uration, they transmit the proposal to all servers to savebandwidth by having servers exchanging only a digest ofeach transaction. The performance is explained by thefact that HotStuff decides only one proposal per consen-sus instance, regardless of the number of submitted trans-actions, which is confirmed by previous observations [51]and [53, Sect.8.3]. By contrast, LLB becomes faster asthe system size increases to outperform HotStuff by 5.6×at n = 90.

Finally, Polygraph is faster at small scale than LLB,because Polygraph’s distributed verification and reliablebroadcast implementations [13] are not accountable, per-forming less verifications. After 40 nodes, Polygraph be-comes slower than LLB because of our optimizations:e.g., its RSA verifications are larger than our ECDSA sig-natures, consume more bandwidth and more CPU.

5.2 Scalability of LLB despite each coali-tion attack

To evaluate LLB under failures, we implemented bothcoalition attacks: the reliable broadcast attack and the bi-nary consensus attack (§3) with f − q = d5n/9e − 1deceitful replicas and q = 0 benign replicas. To disruptcommunications between partitions of honest replicas, weinject random communication delays between partitions

11

based on the uniform and Gamma distributions, and theAWS delays obtained in previously published measure-ments traces [44, 20, 19]. (Deceitful replicas communi-cate normally with each partition.)

Fig. 6(top) depicts the amount of disagreements, as thenumber of distinct proposals decided by honest replicas,under the binary consensus attack. First, we select uni-formly distributed delays between the two partitions withmean as high as 200, 500 and 1000 milliseconds. Then,we select delays following a Gamma distribution with pa-rameters taken from [44, 20] and a distribution that ran-domly samples the fixed latencies previously measuredbetween AWS regions [19]. We automatically calculatethe maximum amount of branches that the size of deceit-ful faults can create (i.e., 3 branches for f − q < 5n/9),we then create one partition of honest replicas for eachbranch, and we apply these delays between any pair ofpartitions.

Interestingly, we observe that our agreement property isscalable: the greater the number of replicas (for the samerelative deceitful ratio), the harder for attackers to causedisagreements. This interesting scalability phenomenonis due to an unavoidable increase of the communicationlatency between attackers as the scale enlarges, whichgives relatively more time for the partitions of honestreplicas to detect the deceitful replicas that have equivo-cated and construct PoFs. This latency increase translatesinto a higher chance of detecting PoFs before deciding,which automatically cancels the upcoming decision andhence limits the number of disagreements. With morerealistic network delays (Gamma distribution and AWSlatencies) that are lower in expectation than the uniformdelays, deceitful replicas can barely generate a single dis-agreement, let alone with an increasing number of repli-cas. This confirms the scalability of our system.

Fig. 6(bottom) depicts the amount of disagreements un-der the reliable broadcast attack. Recall that this attackconsists of having deceitful replicas equivocating whenexpected to reliably broadcast the same proposal, hencesending different proposals to different partitions. We canobserve that the number of disagreement is substantiallyhigher during this attack than during the previous attack,however, it drops faster as the system enlarges, becausethe attackers expose themselves earlier than in the binaryconsensus attack.

34 50 56 59 61 62 63 64 65 66Deceitful faults

048

121620

n=100, 2s uniform delay

0.2 0.5 1 5 10Uniform delay (s)0

12243648 n=100

n=60n=20Di

sagr

eem

ents

Figure 7: Total disagreements (#distinct decisions) per:deceitful faults (top) for a setting with 100 replicas and a2-second uniform delay between partitions, and numberof replicas (bottom) for a variety of uniform delays.

5.3 Disagreements due to failure rates anddelays

We now evaluate the impact of even larger coalitions anddelays on LLB, we measure the number of disagreementsas we increase the deceitful ratios and the partition delaysin a system from 20 to 100 replicas. Note that these delayscould be theoretically achieved with man-in-the-middleattacks, but are notoriously difficult on real blockchainsdue to direct peering between the autonomous systems ofmining pools [23].

Figure 7(top) depicts the number of disagreements asthe coalition size and the number of branches created in-creases with a 2-second uniform delay between partitions.As expected, the larger the coalition, the more disagree-ments up to 23 disagreements for f = d 2n

3 e − 1. Thisis due to deceitful replicas speeding up reliable broadcastby skipping verifications and gathering rapidly the signa-tures for each partition. Although omitted in the figure,the same result occurs for the reliable broadcast attack asthe deceitful ratio increases

While LLB is quite resilient to attacks for realistic butnot catastrophic delays, attackers can wait and try to at-tack when the network collapses for a few seconds be-tween regions. Fig. 7(bottom) shows the amount of pro-posals attackers can disagree on in such a catastrophicscenario, reaching up to 52 disagreeing proposals for auniform delay of 10 seconds between partitions of honestreplicas for the binary consensus attack. Although omit-ted here for the sake of space, the reliable broadcast attackreaches up to 165 disagreeing proposals with a 5-second

12

gamma aws 5001000

10000Delay (ms)

0

20

40

60

80

Tim

e to

det

ect (

s) n=100n=60n=20

gamma aws 5001000

10000Delay (ms)

0

20

40

60

80

Tim

e to

exc

lude

(s)

n=100n=60n=20

gamma aws 5001000

10000Delay (ms)

0

5

10

15

20

25

Tim

e to

inclu

de (s

)

n=100n=60n=20

20 40 60 80 100Number of replicas

0

2

4

6

8

10

Tim

e to

cat

ch u

p (s

)

30 blocks20 blocks10 blocks

Figure 8: Time to detect dn3 e deceitful replicas (left), exclude them (center-left), include new replicas (center-right),per delay distribution and number of replicas; and catch up per number of blocks and replicas (right), with f =d5n/9e − 1.

uniform delay.

5.4 Latencies of block merge and member-ship change

To have a deeper understanding of the cause of LLB de-lays, we measured the time needed to merge blocks andto change the membership by replacing deceitful replicasby new ones.

Blocksize (txs) 100 1000 10000Time (ms) 0.55 4.20 41.38

Table 1: Time to merge locally two blocks for differentsizes with all transactions conflicting.

Table 1 shows the worst-case times to locally mergeblock proposals for different numbers of transactions perblock, assuming all transactions conflict. This is the timetaken in the worst case because replicas can merge pro-posals that they receive concurrently (i.e., without haltingconsensus). It is clear that this time to merge blocks lo-cally is negligible compared to the time it takes to run theconsensus algorithm.

Figure 8 shows the time to detect fd ≥ dn3 e deceit-ful replicas (left), to run the exclusion consensus (center-left), and to run the inclusion consensus (center-right), fora variety of delays and numbers of replicas. The time todetect reflects the time from the start of the attack untilhonest replicas detect the attack: If the first dn3 e deceitfulreplicas are forming a coalition together and creating themaximum amount of branches, then the times to detectthe first deceitful replica and the first dn3 e deceitful repli-cas overlap entirely, as they are detected cross-checkingconflicting certificates. (We detect all at the same time.)

Moreover, the times to detect, exclude, and include in-crease as the communication delays increase; the time toexclude (57 seconds) is significantly larger than to include(21 seconds). This is due to the proposals of the exclusionconsensus carrying PoFs and leading replicas to execute atime consuming cryptographic verification. With shortercommunication delays, performance becomes practical.Finally Figure 8 (right) depicts the time to catch up de-pending on the number of proposals (i.e., blocks). Asexpected, this time increases linearly with the number ofreplicas, due to the catchup requiring to verify larger cer-tificates.

6 Application to Zero-Loss Pay-ment System

In this section, we describe how LLB can be used toimplement a zero-loss payment system where no honestreplica loses any coin. The key idea is to request the con-sensus replicas to deposit a sufficient amount of coins inorder to spend, in case of an attack, the coins of deceitfulreplicas to avoid any honest replica loss. In order to mea-sure the expected impact of a coalition attack succeedingwith probability ρ in forking LLB by leading a consen-sus to disagreement, we first need to make the followingassumptions:

1. Fungible assets. We assume that users can transferassets (like coins) that are fungible in that one unit isinterchangeable and indistinguishable from anotherof the same value. An example of a fungible assetis a cryptocurrency. We assume that an honest clientdoes not issue conflicting transactions, i.e., concur-

13

rent transactions that withdraw from the same ac-count.

2. Deposit refund per block. To limit the impactof one successful double spending on a block, weupper-bound the amount of deposited coins that canbe refunded per block. We denote the maximumamount of deposit per account that can be refundedper block as `. This limit implies that transfer-ring any amount ¢ can be done in one block butthere should be an associated deposit of the sameamount that cannot be refunded in less thanm = ¢/`blocks—a portion of the deposit is refunded in eachof these blocks. We denote m as the depth of the at-tack. We show below that we ensure zero loss whenm ≥ aρ

1−ρ , where a is the number of branches.

3. Network control restriction. Once Byzantine repli-cas select the disjoint subsets of honest replicas todouble spend (i.e. the partitions), we need to preventByzantine replicas from communicating infinitelyfaster than honest replicas in different partitions.More formally, let X1 (resp. X2) be the randomvariables that indicate the time it takes for a mes-sage between two replicas within the same partition(resp. two honest replicas from different partitions).We have E(X1)/E(X2) > ε, for some ε > 0.Note that the definition of X1 also implies that it isthe random variable of the communication time ofeither two honest replicas of the same partition ortwo Byzantine replicas. Notice that this probabilis-tic synchrony assumption is similar to that of Bit-coin and other Blockchains that guarantee exponen-tially fast convergence, a result that is not different inLLB under the same assumptions. Notwithstanding,we show in the following an analysis that focuses onthe attack on each consensus iteration, considering asuccessful disagreement if there is a fork in a singleconsensus instance, even for a short period of time.

Theoretical analysis. We show that attackers alwaysfund at least as much as they steal, leading to zero loss.We consider that a membership change starts before a dis-agreement occurs or does not start, which is safer than thegeneral case. Out of one attack attempt, the attacker maygain G coins by forking or lose P coins as a punishment

from the system. In case of a successful attack, the sys-tem compensates the coin losses using the deposit coins.The attack represents a Bernoulli trial that succeeds withprobability ρ, which can be derived from ε. The randomvariable Y measures the number of attempts for the attackto be successful and follows a geometric distribution withmean E(Y ) = 1−ρ

ρ , where ρ = 1 − ρ is the probabilitythat the attack fails.

The gain of an attack depends on the amount of coinsof the coalition spent in a branches of a fork. That is, themaximum gain of a successful attack is G · (a−1), whereG is the sum of all assets of the attacker coalition. Wedefine the function α : [0,m] ∩ Z → [0, 1] that, given anamount of consensus instances, returns the percentage oftotal gain that is refunded in that amount of instances. Weconsider that an attack tries to steal an amount divisibleby `, thus α(i) = i

m .The expected gain and punishment for the attackers in

a disagreement attempt are as follows:

G(ρ) =(a− 1) ·(P(Y > m) ·G +

m∑i=0

P(Y = i) · α(i) ·G),

P(ρ) =

m∑i=0

P(Y = i) · (1− α(i)) ·G.

We refer to the expected deposit flux per attack attemptas the difference ∆ = P(ρ) − G(ρ) between the punish-ment and the gain from an attack. Therefore, the depositflux is:

∆ =

(1−a·

( m∑i=0

P(Y = i)·α(i)+ρm+1))

G = g(a, ρ,m)G.

If ∆ < 0 then a cost of G(ρ) − P(ρ) is incurred to thesystem, otherwise the punishment is enough to fund thedeposit, so we have to ensure that ∆ ≥ 0, hence we areinterested in g(a, ρ,m) ≥ 0:

1−a·(

(1− ρ)

m·ρ(mρm+1 − (m+ 1)ρm + 1

)(p− 1)2

)−a·ρm+1 ≥ 0.

And choosing m such that m ≥ aρ1−ρ we have:

1−(mρm+1−(m+1)ρm+1

)−a·ρm+1 ≥ 0 ⇐⇒ 1 ≥ ρ

which always holds for any a ≥ 2, hence guaranteeingthat ∆ ≥ 0. We can conclude that

14

10 20 30 40 50 60 70 80 90 100Number of replicas

50

25

0

25

50

75

Depo

sit fl

ux (%

) 500ms, l=10%1000ms, l=10%500ms, l=10%, rbbcast1000ms, l=10%, rbbcast

Figure 9: Expected deposit flux ∆ per uniform delay for` = 10% coins to be refunded per block. ∆ ≥ 0 indicateszero-loss and ‘rbbcast’ indicates the reliable broadcast at-tack.

LLB implements a zero-loss payment system if the to-tal amount to be refunded per account from the deposit ina consensus instance is limited to ` ≤ ¢ 1−ρ

aρ , with m = ¢l ,

since the remaining deposited funds will always be at leastas much as what the attackers manage to steal. If ¢ = C

then the attackers cannot even steal the entire circulatingsupply, if ` and m are chosen accordingly. The completeproof is deferred to the full version.

Theorem 6.1 (Zero Loss Payment System). Given ρ theprobability of success of an attack, if m ≥ aρ

1−ρ then LLBimplements a zero loss payment system.

Simulating the depth and width of a coalition attack.Considering a deceitful ratio δ = f−q

n , one can derivethe maximum number of branches from the bounda ≤ n−(f−q)

d2n/3e−(f−q) [48]. Thus, given a deceitful ratioδ, it is possible to estimate the maximum probability ofan attack succeeding for a particular depth of an attack,or the other way around. For example, for a deceitfulratio of δ = 0.5, a = 3, and for a probability ρ = 0.5,fixing the depth of the attack to m = 3 blocks alreadyguarantees zero-loss, but increasing ρ = 0.9 requiresto take at least m = 27 blocks to finish. It is easy tonotice that, whereas m increases polynomially with ρ, itincreases exponentially as the deceitful ratio δ approachesthe asymptotic limit 2/3, since the number of branchesalso increases exponentially fast, with m = 6 blocks forδ = 0.6, while m = 14 for δ = 0.64 or m = 51 forδ = 0.66, with ρ = 0.5.

Experimental evaluation of the payment system.Taking the experimental results of §5 and based on our

aforementioned theoretical analysis, Figure 9 depicts theexpected deposit flux for a variety of uniform communi-cation delays with f = d5n/9e − 1 where ` is the maxi-mum amount to be refunded per block. Again, we can seethat the expected deposit flux increases with the numberof replicas, confirming that the zero loss property scaleswell. Additionally, small uniform delays yield zero lossat larger values `. Nonetheless, although omitted in thefigure, all delays shown lead to zero loss for at least 90replicas with ` ≤ 50% of the total circulating supply. Thismeans that if LLB simply returns half the deposit immedi-ately after deciding a transaction, and the other half afterthe following block is decided, no honest user will loseany asset whatsoever, even if some attacks succeed. Al-though omitted in the figure, our experiments showed thateven for a uniform delay of 5 or 10 seconds, restricting` = 1% still yields a non-negative deposit flux in the caseof a binary consensus attack, while the reliable broadcastattack requires ` = 0.3%. Nevertheless, if the networkperforms normally, LLB is expected to be gainful for alarge value of f , and to actually benefit from attackers(i.e., obtaining more from punishing them than they cansteal by attacking).

7 Related Work

Several works have tried to circumvent the upper boundon the number of Byzantine failures [47] to reach agree-ment.

Permissionless blockchains. LLB is open permis-sioned as it requires an initial list of replicas that havethe permission to propose to the consensus. This is dif-ferent from permissionless blockchains [45, 52] whosereplicas can solve a cryptopuzzle on-the-fly to proposea block to the consensus. LLB builds upon membershipchange or view change (e.g., [34]) in order to change thepermissions at runtime. In-production blockchains, likeHyperledger Fabric [3], also support dynamic member-ship3. Other membership change protocols adjust permis-sions without disrupting the blockchain service with syn-chrony [46, 1] or partial synchrony [50, 7].

3https://hyperledger-fabric.readthedocs.io/en/latest/glossary.html#dynamic-membership.

15

Slashing. Slashing stakes has become popular to dis-incentivize blockchain participants to misbehave. TheCasper [11] algorithm incurs a penalty in case of dou-ble votes. While safe, Casper does not ensure consen-sus termination when f < n/3 as one replica can alwaysrestart a new consensus instance in a later layer of its di-rected acyclic graph [11]. Tendermint [9] aims at slash-ing replicas, but its consensus is not accountable [10].SUNDR [37] assumes honest clients that can communi-cate directly in order to detect some Byzantine failures.Polygraph [13] solves the accountable consensus but doesnot provide slashing.

Weak guarantees. It was shown that consensus is nec-essary to implement a cryptocurrency when differentreplicas can issue conflicting transactions [30]. Assum-ing that they cannot, one may simply use reliable broad-cast [17]. Zeno [48] guarantees eventual consistency bydecoupling requests into weak (i.e., requests that may suf-fer reordering) and strong requests. It provides availabil-ity in periods where f goes beyond n/3 by committingonly weak requests, that can later be reordered if a mergeprocedure was required. Note that LLB could not be builtupon Zeno because Zeno requires wrongly ordered trans-actions to be rolled back, whereas blockchain transac-tions can have irrevocable side effects like the shipping ofgoods to the buyer. BFT2F [38] offers fork* consistency,which forces the adversary to keep correct clients in onefork only, while also allowing accountability. These pro-posals do not aim at excluding deceitful replicas.

Failure model. Distributed systems within closed net-works usually consider that omission faults (omittingmessages) are more frequent than commission faults(sending wrong messages) [16, 32, 40]. Eve [32] upper-bounds the number of commission faults to respond cor-rectly to database requests. Depot [40] offers a cloud ser-vice with a stronger guarantee than fork-join consistencywhen all faults are omission faults. This is because tradi-tional datacenters build upon an isolated network whoserare commission faults can be due to disk errors [16]. TheBAR model [2], and its Byzantine, altruistic, rational clas-sification is motivated by multiple administrative domainsand corresponds better to the blockchain open networkswithout distinguishing benign from deceitful faults.

Strengthening fault tolerance. Various non-blockchain systems already refined the types of failuresto strengthen guarantees. Upright [16] proposes a systemthat supports n = 2u+ r+ 1 faults, where u and r are thenumbers of commission and omission faults, respectively.They can either tolerate n/3 commission faults or n/2omission faults. Flexible BFT [42] offers a failuremodel and theoretical results to support d2n/3e − 1alive-but-corrupt replicas. An alive-but-corrupt replicaonly behaves maliciously when it can violate safety, butit behaves correctly otherwise. This assumption is toostrong for our needs: for example, it is difficult for areplica to detect whether its coalition is large enoughfor its attack to succeed. Some hybrid failure modelstolerate crash failures and Byzantine failures but preventByzantine failures from partitioning the network [39].Others aim at guaranteeing that well-behaved quorumsare responsive [40] or combine crash-recovery withByzantine behaviors to implement reliable broadcast [4].

8 ConclusionIn this paper, we proposed an open permissionedblockchain LLB whose membership change makes it con-verge, despite d2/3e − 1 deceitful failures, to a statewhere it can solve consensus deterministically. We showthat the cost of this convergence does not prevent LLBfrom scaling and performing almost as fast as an efficientblockchain, even outperforming a recent state machinereplication in a geodistributed network.

AcknowledgmentsThis research is supported under Australian ResearchCouncil Discovery Projects funding scheme (project num-ber 180100496) entitled “The Red Belly Blockchain: AScalable Blockchain for Internet of Things”.

References[1] Ittai Abraham, Dahlia Malkhi, Kartik Nayak, Ling

Ren, and Alexander Spiegelman. Solida: Ablockchain protocol based on reconfigurable byzan-

16

tine consensus. In OPODIS, pages 25:1–25:19,2017.

[2] Amitanand S. Aiyer, Lorenzo Alvisi, AllenClement, Mike Dahlin, Jean-Philippe Martin, andCarl Porth. BAR fault tolerance for cooperative ser-vices. In ACM SOSP, pages 45–58, 2005.

[3] Elli Androulaki, Artem Barger, Vita Bortnikov,Christian Cachin, Konstantinos Christidis, An-gelo De Caro, David Enyeart, Christopher Fer-ris, Gennady Laventman, Yacov Manevich, Srini-vasan Muralidharan, Chet Murthy, Binh Nguyen,Manish Sethi, Gari Singh, Keith Smith, Alessan-dro Sorniotti, Chrysoula Stathakopoulou, MarkoVukolic, Sharon Weed Cocco, and Jason Yellick.Hyperledger fabric: A distributed operating systemfor permissioned blockchains. In ACM EuroSys,pages 30:1–30:15, 2018.

[4] Michael Backes and Christian Cachin. Reliablebroadcast in a computational hybrid model withByzantine faults, crashes, and recoveries. In IEEE/I-FIP DSN, pages 37–46, 2003.

[5] Michael Ben-Or, Ran Canetti, and Oded Goldreich.Asynchronous secure computation. In ACM STOC,pages 52–61, 1993.

[6] Michael Ben-Or, Boaz Kelmer, and Tal Rabin.Asynchronous secure computations with optimal re-silience (extended abstract). In ACM PODC, pages183–192, 1994.

[7] Alysson Bessani, Eduardo Alchieri, Jo ao Sousa,Andre Oliveira, and Fernando Pedone. From byzan-tine replication to blockchain: Consensus is onlythe beginning. In IEEE/IFIP DSN, pages 424–436,2020.

[8] Gabriel Bracha. Asynchronous Byzantine agree-ment protocols. Inf. Comput., 75(2):130–143, 1987.

[9] Ethan Buchman. Tendermint: Byzantine fault toler-ance in the age of blockchains, 2016. MS Thesis.

[10] Ethan Buchman, Jae Kwon, and Zarko Milosevic.The latest gossip on BFT consensus. Technical Re-port 1807.04938, arXiv, 2018.

[11] Vitalik Buterin and Virgil Griffith. Casperthe friendly finality gadget. Technical Report1710.09437v4, arXiv, Jan 2019.

[12] Miguel Castro and Barbara Liskov. Practical Byzan-tine fault tolerance and proactive recovery. ACMTrans. Comput. Syst., 20(4):398–461, November2002.

[13] Pierre Civit, Seth Gilbert, and Vincent Gramoli.Polygraph: Accountable Byzantine consensus. InWorkshop on Verification of Distributed Systems(VDS’19), Jun 2019.

[14] Pierre Civit, Seth Gilbert, and Vincent Gramoli.Brief announcement: Polygraph: Accountablebyzantine agreement. In DISC, volume 179, pages45:1–45:3. Springer, Oct 2020.

[15] Allen Clement, Flavio Junqueira, Aniket Kate, andRodrigo Rodrigues. On the (limited) power of non-equivocation. In ACM PODC, pages 301–308, 2012.

[16] Allen Clement, Manos Kapritsos, Sangmin Lee,Yang Wang, Lorenzo Alvisi, Michael Dahlin, andTaylor Riche. Upright cluster services. In ACMSOSP, pages 277–290, 2009.

[17] Daniel Collins, Rachid Guerraoui, Jovan Ko-matovic, Petr Kuznetsov, Matteo Monti, MatejPavlovic, Yvonne Anne Pignolet, Dragos-AdrianSeredinschi, Andrei Tonkikh, and AthanasiosXygkis. Online payments by merely broadcastingmessages. In IEEE/IFIP DSN, pages 26–38, 2020.

[18] Tyler Crain, Vincent Gramoli, Mikel Larrea, andMichel Raynal. DBFT: Efficient leaderless Byzan-tine consensus and its applications to blockchains.In IEEE NCA, 2018.

[19] Tyler Crain, Christopher Natoli, and VincentGramoli. Evaluating the Red Belly Blockchain.Technical Report 1812.11747, arXiv, 2018.

[20] Mark E Crovella and Robert L Carter. Dynamicserver selection in the Internet. In IEEE HPCS,1995.

17

[21] Sisi Duan, Michael K. Reiter, and Haibin Zhang.BEAT: asynchronous BFT made practical. In ACMCCS, pages 2028–2041, 2018.

[22] Cynthia Dwork, Nancy Lynch, and Larry Stock-meyer. Consensus in the presence of partial syn-chrony. J. ACM, 35(2):288–323, April 1988.

[23] Parinya Ekparinya, Vincent Gramoli, and GuillaumeJourjon. Impact of man-in-the-middle attacks onethereum. In IEEE SRDS, 2018.

[24] Zachary Amsden et al. The Libra blockchain. Tech-nical report, Calibra, 2019. Revised version ofSeptember 25.

[25] Ittay Eyal, Adem Efe Gencer, Emin Gun Sirer,and Robbert van Renesse. Bitcoin-NG: A scalableblockchain protocol. In USENIX NSDI, pages 45–59, 2016.

[26] Ittay Eyal and Emin Gun Sirer. Majority is notenough: Bitcoin mining is vulnerable. Commun.ACM, 61(7):95–102, June 2018.

[27] Michael J. Fischer, Nancy A. Lynch, and Michael S.Paterson. Impossibility of distributed consensuswith one faulty process. J. ACM, 32(2):374–382,1985.

[28] Yossi Gilad, Rotem Hemo, Silvio Micali, GeorgiosVlachos, and Nickolai Zeldovich. Algorand: Scal-ing Byzantine agreements for cryptocurrencies. InACM SOSP, pages 51–68, 2017.

[29] Guy Golan-Gueta, Ittai Abraham, Shelly Grossman,Dahlia Malkhi, Benny Pinkas, Michael K. Reiter,Dragos-Adrian Seredinschi, Orr Tamir, and AlinTomescu. SBFT: a scalable decentralized trust in-frastructure for blockchains. In IEEE/IFIP DSN,pages 568–580, 2019.

[30] Rachid Guerraoui, Petr Kuznetsov, Matteo Monti,Matej Pavlovic, and Dragos-Adrian Seredinschi.The consensus number of a cryptocurrency. In ACMPODC, pages 307–316, 2019.

[31] A. Haeberlen, P. Kouznetsov, and P. Druschel. Peer-review: Practical accountability for distributed sys-tems. In ACM SOSP, pages 175–188, 2007.

[32] Manos Kapritsos, Yang Wang, Vivien Quema, AllenClement, Lorenzo Alvisi, and Mike Dahlin. Allabout eve: Execute-verify replication for multi-coreservers. In USENIX OSDI, pages 237–250, 2012.

[33] Ramakrishna Kotla, Lorenzo Alvisi, Mike Dahlin,Allen Clement, and Edmund Wong. Zyzzyva: Spec-ulative Byzantine fault tolerance. In ACM SOSP,pages 45–58, 2007.

[34] Leslie Lamport, Dahlia Malkhi, and Lidong Zhou.Reconfiguring a state machine. SIGACT News,41(1):63?73, March 2010.

[35] Leslie Lamport, Robert Shostak, and MarshallPease. The Byzantine generals problem. ACMTrans. Program. Lang. Syst., 4(3):382–401, July1982.

[36] Leslie Lamport, Robert Shostak, and MarshallPease. The Byzantine generals problem. ACMTrans. Program. Lang. Syst., 4(3):382–401, 1982.

[37] Jinyuan Li, Maxwell Krohn, David Mazieres, andDennis Shasha. Secure untrusted data repository(sundr). In USENIX OSDI, page 9, 2004.

[38] Jinyuan Li and David Mazieres. Beyond one-thirdfaulty replicas in Byzantine fault tolerant systems.In USENIX NSDI, page 10, 2007.

[39] Shengyun Liu, Paolo Viotti, Christian Cachin,Vivien Quema, and Marko Vukolic. XFT: practicalfault tolerance beyond crashes. In USENIX OSDI,pages 485–500, 2016.

[40] Marta Lokhava, Giuliano Losa, David Mazieres,Graydon Hoare, Nicolas Barry, Eli Gafni, JonathanJove, Rafał Malinowsky, and Jed McCaleb. Fast andsecure global payments with stellar. In ACM SOSP,pages 80–96, 2019.

[41] Loi Luu, Viswesh Narayanan, Chaodong Zheng,Kunal Baweja, Seth Gilbert, and Prateek Saxena. Asecure sharding protocol for open blockchains. InACM CCS, pages 17–30, 2016.

[42] Dahlia Malkhi, Kartik Nayak, and Ling Ren. Flex-ible Byzantine fault tolerance. In ACM CCS, pages1041–1053, 2019.

18

[43] Andrew Miller, Yu Xia, Kyle Croman, Elaine Shi,and Dawn Song. The honey badger of BFT proto-cols. In ACM CCS, pages 31–42, 2016.

[44] Amarnath Mukherjee. On the dynamics and sig-nificance of low frequency components of Internetload. Technical Report MS-CIS-92-83, Universityof Pennsylvania, 1992.

[45] Satoshi Nakamoto. Bitcoin: a peer-to-peerelectronic cash system, 2008. http://www.bitcoin.org.

[46] Rafael Pass and Elaine Shi. Hybrid consensus: Ef-ficient consensus in the permissionless model. InDISC, pages 39:1–39:16, 2017.

[47] Marshall Pease, Robert Shostak, and Leslie Lam-port. Reaching agreement in the presence of faults.J. ACM, 27(2):228–234, 1980.

[48] Atul Singh, Pedro Fonseca, Petr Kuznetsov, RodrigoRodrigues, Petros Maniatis, et al. Zeno: Eventuallyconsistent Byzantine-fault tolerance. In USENIXNSDI, pages 169–184, 2009.

[49] Pierre Tholoniat and Vincent Gramoli. Formal ver-ification of blockchain byzantine fault tolerance. In6th Workshop on Formal Reasoning in DistributedAlgorithms (FRIDA’19), Oct 2019.

[50] Guillaume Vizier and Vincent Gramoli. ComChain:A blockchain with byzantine fault tolerant reconfig-uration. Concurrency and Computation, Practiceand Experience, 32(12), Oct 2020.

[51] Gauthier Voron and Vincent Gramoli. Dispel:Byzantine SMR with distributed pipelining. Tech-nical Report 1912.10367, arXiv, Dec. 2019.

[52] Gavin Wood. Ethereum: A secure decentralizedgeneralized transaction ledger, 2015.

[53] Maofan Yin, Dahlia Malkhi, Michael K. Reiter,Guy Golan Gueta, and Ittai Abraham. HotStuff:BFT consensus in the lens of blockchain. Techni-cal Report 1803.05069, arXiv, July 2019. Version 6(accessed 21 May 2019).

[54] Maofan Yin, Dahlia Malkhi, Michael K. Reiter,Guy Golan Gueta, and Ittai Abraham. HotStuff:BFT consensus with linearity and responsiveness. InACM PODC, pages 347–356, 2019.

[55] Mahdi Zamani, Mahnush Movahedi, and MarianaRaykova. RapidChain: Scaling blockchain via fullsharding. In ACM CCS, pages 931–948, 2018.

19