What is Distributed Consensus?
In my previous post I made the claim that blockchain technology by itself was not particularly revolutionary, since it’s basically a database with some special rules.
What makes a blockchain special is how it can be used to maintain a trusted public record of transactions without the need for a trusted central authority to verify those transactions.
The way this untrusted-but-valid agreement system is handled is called “distributed consensus”, which is what really makes blockchain technology interesting.
What is distributed consensus?
In order for a blockchain to be useful as a distributed database, there needs to be a way to add new information into it.
In order for new information to be added, at least one of the participants in the blockchain network has to make a proposal on what the new information addition should be.
For example, if a group of people are trying to decide where to have lunch, at least one person in the group must propose one or more restaurant options for the group to decide on. Without at least one proposal, no decision on where to eat can be made.
Going back to my explanation of a blockchain, at least one of the parties maintaining a local copy of the blockchain must propose a new block addition for the rest of the group to decide on.
This presents a conundrum. How do you ensure the party who proposed the block is not trying to game the system in their favor?
In the lunch example, the people in the group know each other and the stakes are low, so it really doesn’t matter who makes the proposal.
In a distributed blockchain network, there’s no way of knowing who the party making the new block proposal is and therefore no way of knowing whether or not this party is acting with good intent.
The stakes are potentially a lot higher, since the blockchain records transfers of ownership of digital assets with real world value, and that record cannot be changed once it gets added to the blockchain.
The problem of achieving a trusted consensus in an untrusted system is known as the Byzantine Generals’s Problem. I encourage you to click the link and read the Wikipedia entry, but I’ll give you a quick and dirty summary:
Imagine you are one of several medieval generals surrounding an enemy city. You and the rest of the generals need to collectively decide to attack or retreat, because a partial attack will not succeed.
Although the other generals are your peers, they are also your competitors for favors from the king, so you don’t fully trust them and they don’t fully trust you.
The only way you can communicate with them is via messengers, and you can’t trust the integrity of the messengers either.
In this kind of scenario, how do you decide whether to attack or retreat? How do you know if the information you’re receiving from the messengers is reliable?
Even if the messengers are reliable, how do you know if the other generals aren’t plotting to trick you into attacking by yourself while they retreat?
This is not an easy problem to solve.
In the context of a globally distributed blockchain network, the generals are the computers and the messengers are the packets of information passed between them. The decision they need to agree on is whether or not a new block is a valid addition to the blockchain.
How do participants in a globally distributed blockchain network come to an agreement on what constitutes a valid addition while being resilient enough to reject invalid or malicious additions?
The answer to this question is “distributed consensus”.
Distributed consensus is the process by which everyone who relies on the integrity of the blockchain reaches agreement on what goes into it, without having to trust or even know each other.
This field of computer science is still evolving, but there are currently two commonly accepted approaches to achieving distributed consensus – Proof of Work (PoW) and Proof of Stake (PoS).
What is proof of work?
Going back to the lunch example, anyone in the group can propose a restaurant option. Maybe this group meets for lunch on a regular basis, and maybe Bob always makes the lunch proposals.
The other group members might be fine with this arrangement, since the stakes are low and Bob always knows the best trendy new places to each lunch.
However, maybe Alice would like to suggest a lunch option every now and then because she gets tired of eating at all the trendy expensive places and just wants a simple bowl of soup.
But she can’t! Bob always dominates the lunch discussion because he likes the attention, so Alice can never bring up her suggestion.
This lack of fairness in making proposals is a serious problem with any system that relies on the goodwill of all participants, but it’s especially problematic in a system where there is no central authority available to enforce fairness. A blockchain network is exactly this kind of system.
If someone like Bob was participating in a blockchain network, he could potentially dominate the block selection process and favor the addition of blocks that somehow benefited him.
This kind of behavior is considered malicious intent by a bad actor.
Proof of work is one way to neutralize malicious intent by democratizing the block selection process. This is done by making everyone in the network (including Bob!) compete for the right to add the next block by solving a very hard puzzle.
The difficulty of the puzzle is automatically adjusted up or down based on the number of participants trying to solve the puzzle. The party that finds the correct solution to the puzzle wins the right to propose the next block for addition to the blockchain.
To claim that right, the winner must present their solution to the puzzle to the rest of the network. All other participants will verify the solution and review the newly proposed block. As long as they agree the information stored in the block is valid, the new block gets added to the blockchain.
Incidentally, this process is called “mining” blocks, and the parties competing for the right to create new blocks are called “miners”.
Once confirmed, the new block is added to everyone’s local copy of the blockchain and the competition starts over for the next block.
The puzzles are intentionally designed to be difficult to solve. However, once solved the solution is trivially easy to verify.
If this sounds counterintuitive, think of a Sudoku puzzle. A Sudoku puzzle takes a lot of time to solve, but once solved it’s trivially easy to verify the solution is correct just by looking at the numbers.
Going back to the lunch example, the group could institute a policy where every participant has to solve a Sudoku puzzle in order to make a lunch suggestion. The one who first solves the puzzle shows the solution to the rest of the group, and once they verify all the numbers are in the right place, the winner gets to select where they eat.
Returning to blockchains, the amount of computation required to find the right solution prevents any one party from dominating the selection process. The automatic scaling of puzzle difficulty also prevents any party from just adding more computers to try and solve puzzles faster.
Proof-of-Work is an elegant solution to the Byzantine Generals’ Problem, but it has major drawbacks. The biggest issue with the Proof-of-Work approach to distributed consensus is the massive amount of energy consumption required to maintain it.
Since only one party can win each round of the block-selection competition, everyone else’s computation power is wasted in a losing attempt to find the solution.
For a large established network like Bitcoin, this wasted computation power translates into an enormous amount of wasted electricity and excess heat generated by the computers competing for block-selection rights.
Electricity isn’t free and the excess heat has to be dealt with. These two factors translate into huge costs to maintain the Bitcoin network.
You might be wondering why someone would even bother competing for the right to select blocks if most of the energy consumed will be wasted, resulting in higher utility bills.
A network that uses a Proof-of-Work consensus approach typically incentivizes parties to “mine” blocks through the awarding of fees and/or new network tokens that have a real world value, along with the right to propose the next block.
In the case of Bitcoin, a transaction fee is typically set by whoever initiates the transaction. This fee is the amount of money a sender is willing to pay to see his or her transaction picked up by a “miner” and placed in a new block. There is no minimum or maximum amount for a fee, the amount is entirely up to the sender.
Since fees are completely arbitrary, “miners” are incentivized to select transactions with the highest fees for inclusion in a block, in order to maximize their gain if they happen to win a round of the block-selection competition.
They are also incentivized to act in the best interest of the network as a whole, since doing otherwise would jeopardize their gains.
The collection of these fees, coupled with the awarding of new bitcoins, are called mining rewards.
These rewards offset the cost of computers and energy needed to compete in the block-selection competition, and they make bitcoin mining a profitable venture for parties with access to cheap electricity and abundant cooling capacity.
In the lunch example, we could say that the people in the group are incentivized to solve the Sudoko puzzle because the winner gets a free lunch plus a few bucks extra, along with the right to propose where they eat.
Even though incentives make block “mining” profitable, the massive amount of energy wasted in a Proof-of-Work system is still problematic and arguably not sustainable over the long term.
Several alternatives for achieving distributed consensus without requiring huge expenditures of energy has evolved over the last few years, with the most prominent among them being Proof-of-Stake.
What is Proof-of-Stake?
Proof-of-Stake attempts to solve the Byzantine Generals’ Problem by requiring all participants to have a financial stake in the outcome of the vote. Since everyone has “skin in the game”, they are incentivized to act honestly in order to avoid losing their stake.
In the context of a blockchain, the block selection contest I discussed earlier is not won through solving puzzles. Instead, a Proof-of-Stake system is typically set up to select one party every round from a group willing to post all or part of their blockchain assets as stakes.
The selection can be random, or it can be based on a set of factors like a participant’s voting history or the number of blockchain assets that participant controls. This selection method is built into the software that runs the blockchain network, so it is handled automatically without the need for outside control.
The party selected to nominate the next block must propose a valid block in order to avoid losing their stake. Everyone else with a stake gets to vote on the validity of the proposed block, and they are incentivized to vote properly or risk losing their stake.
If the nominated block turns out to be invalid or deemed to be malicious, the nominator forfeits their stake and could be barred from participating in one or more future block selection rounds.
Similar to Proof-of-Work, transaction fees are usually built into Proof-of-Stake systems in order to incentive participation in the block selection process. The winner of the selection round is awarded the transactions fees included in the proposed block, which makes posting a stake profitable.
Returning to the lunch example, let’s say the lunch group is made up of six people. In order to win the right to choose where they eat, each person puts up five dollars as a stake.
For the sake of keeping this example simple, let’s say each person is represented by a number from 1 to 6. They ask a stranger on the street to roll a die for them to decide which person selects the restaurant.
The die rolls a 3, which is Jane’s number. She selects a restaurant, and as long as the restaurant has space available to seat them all then she wins the selection round and gets a free lunch out of the deal (we’ll just say the restaurant comps her lunch as a kickback for bringing in a large group).
If she selects a restaurant that doesn’t have space to seat them, she loses her five bucks and another selection round takes place which she doesn’t get to participate in.
In a blockchain network, the stakes are denominated in network coins. Every participant in the block selection contest puts up some number coins as a stake in order to win the right to select the next block.
Ethereum is currently the second most well known cryptocurrency after Bitcoin, and although the project currently uses a Proof-of-Work system, they plan to move to a Proof-of-Stake system in the near future.
A Proof-of-Stake system is much more efficient than a Proof-of-Work system in terms of energy costs required to maintain the network because no one has to perform a lot of wasteful computation. Instead, they only have to put up a stake to participate in the system.
If you’ll recall, Bob used to dominate the lunch selection process because he likes being the center of attention. Let’s say Bob is secretly bitter that he no longer dominates the lunch selection process.
What’s to stop Bob from colluding with Jane, Steve, and Carol to control 4 out of the 6 votes and have them vote his way?
This is called a monopoly problem, or a 51% attack. What this means is a bad actor like Bob attempts to control a majority of the resources involved in block selection in order to influence block selection in his favor.
In Bob’s case, the other people in the group are unlikely to cooperate with him without an incentive that is greater than the incentive they receive by participating honestly in the process (free lunch!).
Bob will likely have to pay the stakes for Jane, Steve, and Carol in addition to his own, and probably promise them all a free lunch to get them to go along with him. So now Bob is out $25 in stakes plus the cost of three lunches every time the group gets together for lunch.
Plus, if the others find out the lunch group is finished, putting an end to everyone’s participating and rendering Bob’s takeover attempt a big expensive failure.
The added expense of trying to control the network acts as a deterrent to any one party dominating the selection process.
The same concept applies to a blockchain network that uses a Proof-of-Stake system. Any bad actor would have to buy at least 51% of the market cap of the cryptocurrency in order to control the system. The expense of doing so would make it in their best interest to act honestly in order to avoid losing their entire investment in the system.
As mentioned previously, distributed consensus is an area of computer science that is still relatively new and evolving.
Proof-of-Work and Proof-of-Stake are only two of the many different proposals for achieving distributed consensus, but right now they are the most widely adopted.
If you have any questions about Proof-of-Work, Proof-of-Stake, or anything else covered in this post feel free to comment below!
About the Author
I'm a software engineer with a background in presenting simple explanations of complex topics. My goal is to explain the technologies shaping our world in terms that my non-tech friends and family will understand.