The Bitcoin protocol is in constant need of upgrade. Either to add new features, fix critical bugs, or improve scalability. The question is how best to upgrade the protocol of an already-deployed multi-billion dollar currency/payment system without disrupting everything. To do so we really only have two options ― softforks and hardforks. We’ll get into the difference between the two in a moment. To date, all planned protocol upgrades have been done via softfork and none via hardfork. Naturally, as with all things in Bitcoin, this has become a point of contention.
Up until recently I was firmly in the softfork camp. If we can upgrade the protocol rather seamlessly without causing any noticeable disruption, then why not do all upgrades as a softfork? Looking back on it, this was a rather naive view to have. I never really thought through the consequences of both types of forks and just sort of went along with what everyone else was telling me. I will get to my criticisms of softforks in a moment, but first a refresher.
What’s the difference?
The Bitcoin blockchain is record of all transactions from the start of bitcoin to today. Transactions are added to the blockchain in batches (“blocks”) creating a linear chain of blocks.
If some of the nodes/miners in the network were to upgrade their software in an incompatible way, blocks created by upgraded miners would be rejected by nodes/miners that have not upgraded. This is called a hard fork.
In the extreme case, suppose 51% of miners upgraded and 49% did not, we would see a permanent divergence in the blockchain, essentially two competing transaction histories, starting at the point at which some miners upgraded.
To give a concrete example, consider the current block size debate. The current protocol rejects blocks larger than 1 megabyte. If some miners were to upgrade their software to create 2 megabyte blocks, those blocks would be rejected by non-upgraded nodes, while accepted by upgraded nodes.
This kind of “even split” would be pretty disastrous for Bitcoin. Not necessarily from a technical perspective, eventually one side of the fork would likely win out over the other, but it would result in a firestorm of bad PR which would set Bitcoin back many years.
For this reason if protocol upgrades were to be done via a hardfork, some really high threshold of support (like 95%) would be required for activation of the new rules. This would avoid the doomsday scenario. If 5% refused to upgrade, it would largely be a non-event as we can always expect a few curmudgeons. And in all likelihood, the remaining 5% would upgrade after activation as it would be pointless to continue on a 5% fork.
So what’s a softfork? Basically, it’s change to the protocol that is compatible with the existing rules, but just more restrictive. A perfect example would be the block size again. Remember nodes currently accept blocks up to 1 megabyte in size. But what if a rule change was put in place setting the maximum block size at 500 kb? These smaller blocks would be accepted by non-upgraded nodes because, after all, they are less than 1 mb. But wouldn’t this also create a fork in the chain if some miners/nodes accept 1 mb blocks and others don’t? Yes, but only temporarily.
Consider the following example. Suppose this small block softfork is put in place and 51% of miners upgrade and 49% do not. It’s likely that, due to randomness, the non-upgraded nodes will periodically find a string of blocks in row. In this example, they find three.
Since these blocks are all over the new 500 kb limit, the upgraded nodes reject them and continue trying to extend the last valid block they saw. Since the upgraded nodes are in a majority, it’s mathematically certain that the upgraded nodes will eventually extend their 500 kb chain longer than the 1 mb chain.
Since the protocol rules state that, when confronted with two valid chains, the longest chain should be treated as valid, the non-upgraded miners will stop working to extend the 1 mb chain and switch over to the 500 kb chain.
The three 1 mb blocks that were mined (and all the transactions they contain) will be ignored. When nodes switch chains like this and reject previously valid blocks it’s called a blockchain reorganization. Note that while softforks may cause periodic forks and reorgs, as long as the upgraded miners are in a majority, the blockchain will (eventually) converge into a single chain. This is the source of claimed advantage that softforks have over hard forks ― no risk of permanent chain split.
Or course, this isn’t exactly true. There still can be a permanent split in the blockchain if the upgraded miners are in a minority. Then the blockchain will permanently diverge just like the case of a hard fork. For this reason, the softfork activation threshold is also set very high (95%) to guard against the risk of a permanent split.
Another claimed benefit of softforks is that, because the chain converges, full node operators do not necessarily need to update their software. Since it doesn’t require every single full node operator to upgrade or risk being forked off the network (think of being in the 5% in the hardfork example), softforks are easier to roll out.
What’s wrong with softforks?
As I just described it, softforks sound relatively benign. They more or less guarantee that the blockchain will always converge, avoid the need for everyone to upgrade their software, and enable fast rollouts. So what’s wrong?
Well, in essence they only accomplish this magic feat of a protocol upgrade without disruption by reducing the security of every non-upgraded full node.
How so? Since the protocol rules have changed, your (non-upgraded) node cannot validate the new rules. It simply doesn’t know what the new rules are. In theory your node could receive transactions that are valid by the old rules (and hence accepted by your node), but invalid by the new rules (rejected by upgraded nodes). Any transaction you receive could be either invalid, or built on top of a child transaction that is invalid. Your node can’t tell the difference.
Since you no longer have a validating node, the only way you have to tell if a transaction is valid or not is to wait to see if it confirms. So in essence the softfork transforms your previously fully validating node into a SPV (simplified payment verification) node which uses depth as a proxy for validity.
Now in reality SPV is still pretty secure. I personally think it has been given an undeserved reputation as insecure by the Core devs. So in all probability you are not going to be defrauded by failing to upgrade your node in a softfork. But the point still remains that the reason people go through the effort to run a full node is precisely because they want the security of full validation. If they wanted an SPV node, they could have just run SPV.
I’ve had people tell me that non-upgraded nodes are not as bad as SPV because they at least validate the old rules. But what good is that? In the (rather unlikely) event that you do get attacked and accept coins on an invalid chain, do you think the businesses where you try to spend those coins are going to care that your coins were validated under the old rules? Of course not. Invalid coins are invalid coins just the same.
Maybe an non-upgraded node could be thought to be more secure than SPV because it can’t be tricked into accepting coins created out of thin air. That an attacker would at least need to own the coins he’s purporting to send you. But I don’t even know if that’s true. Depending on the type of softfork, an attacker could fake ownership of other people’s coins, which basically has the same effect.
Finally, note that, going forward, the Core devs plan to have multiple overlapping softforks on continuous roll out. This means that at any given time, the number of fully validating nodes in the network is likely to be dangerously low as most don’t upgrade immediately.
A 51% attack in disguise
So back to my point about the reason people run a full node. People run a node under the belief that they are fully validating all the protocol rules. Even if they don’t pay attention to every minutiae of Bitcoin development, they can rest easy knowing that, with the sole exception of a 51% attack, their node cannot be tricked into accepting invalid transactions.
But then the Core developers come along and coordinate a 51% attack on the network….
To be rather blunt about it, that’s what a softfork is. It’s a developer-coordinated and sanctioned 51% attack on all non-upgraded full nodes. That’s how this seamless protocol upgrade is accomplished.
And the worst part about it is this can be, and has been, done with the consent of very few individuals. In today’s development environment, it only takes an ACK from a handful of Core devs, coupled with support from a handful of mining pool operators and they can push whatever change on the network they want. This puts an enormous power over the protocol in the hands of very few individuals.
But aren’t hardforks still worse?
No not really. As I mentioned above, given an activation threshold of 95%, a hardfork is unlikely to cause any more risk of a permanent chain split than a softfork.
But don’t hardforks still require all full nodes to upgrade whereas softforks don’t? With today’s Bitcoin Core client, yes. But that could be changed by adding a new runtime option. Consider the following:
If you anticipate that your aren’t going to be able to follow the minutiae of bitcoin development and likely won’t upgrade your node immediately, and if you are comfortable with SPV security, you could instruct your node “in the event that a hardfork is detected, drop into SPV mode and continue following the longest chain”. And maybe while it’s at it, your node could send you a text message alerting you of the security change.
Now notice that this is nearly 100% identical to the current softfork functionality. The only difference here is you explicitly opt in to the security reduction, whereas with a softfork it is forced upon you by the Core devs and miners.
Maybe it’s just me, but it seems like consent is always better than no consent. And of course, with a hardfork people can always reject the change by refusing to upgrade, which they can’t do with a softfork.
There’s also a very real sense in which hardforks democratize changes to the Bitcoin protocol. Whereas softforks consolidate decision making into the hands just a few (using a 51% attack to force changes on people without their explicit consent), miners will not hardfork unless the can be certain an economic majority supports the change.
Butt ugly design
Questionable ethics aren’t the only problem with softforks. The protocol was never designed to be upgraded that way. Making changes to the protocol as a softfork requires a series of ugly hacks just to make the change “work”.
For a consumer perspective, these hacks aren’t noticeable, but to other developers they represent warts on an otherwise elegant protocol design. And every softforking upgrade bastardizes the protocol even more.
Maybe the worst offender in this regard is the proposed segregated witness softfork. As one redditor put it, “it requires you to ingest substantial amounts of LSD to envision it as a softfork”.
It’s so bad the Core developers have said “well we can always do a hardfork later on to clean it up”. Excuse me? Can anyone imagine a developer cavalierly proposing to push through an ugly hack into any other multi-billion dollar financial software by saying, “We’ll clean it up later”!? Why not just do it right from the start?
The answer is because the Core devs want to do everything as softforks and avoid hardforks at all cost. I would also point out that there is zero consensus in the community to do segregated witness as a softfork yet it’s an almost certainty that they will. This goes back to my earlier point. Changing the Bitcoin protocol by softfork requires the approval of like 10 people, that’s it. That’s how a segwit softfork will get pushed through despite the objections.
Let’s go back and review the claimed advantages of softforks over hardforks:
Softforks have less of a risk of creating a permanent fork in the chain.
Not really. Both softforks and hardforks would likely use a 95% activation threshold which all but eliminates most of that risk.
Softforks result in a convergence on a single chain, preventing disruption for non-upgraded nodes.
They only do this by 51% attacking those nodes and dropping them to SPV security without their owner’s consent.
Hardforks require all full nodes to upgrade which is very difficult to achieve.
I’ve show how full nodes could be given the option to drop into SPV mode to avoid an immediate upgrade. That’s near 100% identical functionality as a softfork but with actual explicit consent.
Finally hardforks have the benefit over softforks in that they:
- Democratize protocol changes, taking the power to push through a change out of the hands of a tiny few and requiring an actual economic majority to make a change.
- Require the explicit consent of full node operators to any security reduction.
- Allow for a much cleaner protocol design, done right the first time, which will make future changes much easier.
So having got that off my chest, everything above is basically why I changed my opinion of softforks. I no longer see any value in them over hardforks yet I see plenty of faults.