I’m happy that the issue is being discussed more and it seems more people are questioning the appropriateness of softforks. Or at least enough people are questioning them to provoke a response from supporters. I’m open to changing my mind. Bitcoin development isn’t (and has no reason to be) as partisan as, say, political ideology, but Peter’s blog post didn’t convince me.
If you recall from my original post I stated that the primary reason why we are told softforks should be preferred over hardforks is because with softforks the chain converges while with hardforks it forks.
I proceeded to give several reason why I believe that conventional wisdom is wrong:
– Softforks only achieve convergence by dropping all non-upgraded nodes to a lower level of security
– Softforks drop those nodes to a lower level of security without their owner’s consent.
– Softforks often employ ugly hacks to achieve protocol upgrades.
– Softforks allow for a small number of people (usually just a few developers and a few mining pool operators) to change the protocol.
In his post Peter more or less repeats the same line that (paraphrasing) “softforks are safer because the chain converges and hard forks are dangerous because it forks”, without providing much in the way of compelling evidence.
He does say:
[In a hardfork] the blocks from miners adopting the fork are considered invalid by the those who haven’t adopted, because the blocks violate existing rules. So the non-adopting miners build on each others blocks, creating two separate chains. There’s no limit to how long the old chain gets; transactions on the old chain can easily get 6+ confirmations even though they’re considered invalid by the majority of hashing power. This is a serious problem!
Why would we expect this be serious problem? I point out in my previous post that non-upgraded node operators have 3 options:
1) Stick with the old chain (which is likely to be at <5% support)
2) Upgrade to the new version
3) Tell their node to drop into SPV when it detects a hardfork if they don't expect to upgrade right away (this isn't an option currently, but it could be).
Hard to see what the serious problem is! Continuing…
Currently most (all?) SPV wallets don’t even check the block version, so in the above scenario all they see is this:
Are those blocks valid or invalid? Who knows! It depends on whether you happened to connect to nodes that have or have not adopted the new rule. You can of course download new SPV wallet software that explicitly sets a checkpoint for the fork, but until you do that it’s very easy to get ripped off with fake coins.
Ah so the serious problem is not any real threat to consensus but just to SPV wallet users!
I would point out that my post was concerned about softforks dropping the level of security for full node operators. To me, this is a far more important security consideration than SPV wallets. Full node operators run a full node usually because they process high volumes of payments and need/expect maximum security. I suspect that most people who use SPV wallets know that they aren’t getting full node security and hence use it accordingly.
Even if they aren’t, the scenario he points out is fairly unlikely. Note I assumed in my post that hardforks would also use a 95% threshold which also assumes a large economic majority of full node operators support the change (else the miners would be mining coins that are not accepted by the community). It’s theoretically possible that p2p SPV wallets, which make a dozen random outgoing connections to the network, could randomly connect to all non-upgraded nodes and think the old chain is valid, but that’s somewhat unlikely. Every time the wallet starts up or comes back from sleep it makes new connections. If even one of them, just once, connects to an upgraded node, the wallet will follow the upgraded chain from that point forward. And of course wallet software could easily be upgraded to give users the choice: “Version 4 blocks detected, do you want to upgrade to version 4 or stick with version 3?” They don’t do that today because we haven’t been using hardforks to upgrade the protocol.
I would also point out that when people say “softforks are safer than hardforks” you would get the impression that they are referring to the risk to consensus (which is very low if the threshold is set high), but here Peter seems to be only referring to the risk to SPV wallets since that was his only real criticism.
Moving on to his objections to my objections.
But soft forks trick old nodes into degraded security!
All forks degrade security, the question is how badly?
The reason why Bitcoin works at all is because users validate blocks and make sure the rules are being followed; every rule that you aren’t validating and the economic majority is validating is a potential attack.
Secondly, all non-malicious forks explicitly signal that fork is happening by changing the block header version. Good node and wallet soft fork uses that version change to warn the user a fork is happening and let them take appropriate action. Of course, there’s a lot of bad wallet software out there that doesn’t do this, but that’s something this community needs to fix. There’s no trickery involved here – if you don’t want to accept blocks that you might not have validated fully, configure your wallet software to shutdown automatically when it detects an upgrade. If it can’t do that, tell the maintainer to fix that!
Obviously if you upgrade it isn’t a security reduction, but what I spend half my post writing about is how node operators who don’t upgrade have their security reduced without their consent. His solution is if you don’t want the security hit, program your node to shutdown when it detects a softfork. Security problem solved.
I suppose that’s an answer, but it doesn’t strike me as a very good one. With a hardfork you do have another option but I’ll hold off on that until the last objection.
But BIP66’s deployment was a fiasco! Soft forks are dangerous!
When BIP66 triggered deployment it happened a majority of the hashing power was doing validationtionless mining. In short, by mining on top of blocks they hadn’t actually validated they got a head start over miners that took the time to validate first. This is significantly more profitable, because until you start mining on top of the latest block you’re wasting your money.
By the time a majority of hashing power had fixed their nodes, and started validating again, the invalid chain was six blocks ahead of the valid chain.
This meant that SPV clients could have been tricked into accepting an invalid transaction with six confirmations.
So could have non-upgraded full nodes, which I think is an even bigger problem because they are the ones taking thousands/millions of dollars worth of payments. SPV wallets aren’t used for processing large amounts of payments. This vulnerability to non-upgraded nodes happened because of the softfork. Kind of makes my whole point.
In short, during a soft fork, regardless of whether or not your software is upgraded with the new rules, just waiting two or three extra confirmations is pretty safe, so long as the supermajority of miners are validating. That’s not true in a hard fork, and you better upgrade ASAP.
Given the current state of the software you have to upgrade ASAP because you’ll be forked off the main chain if you don’t. But as I pointed out in my last post:
If you anticipate that your aren’t going to be able to follow the minutiae of bitcoin development and likely won’t upgrade your node immediately, and if you are comfortable with SPV security, you could instruct your node “in the event that a hardfork is detected, drop into SPV mode and continue following the longest chain”. And maybe while it’s at it, your node could send you a text message alerting you of the security change.
Now notice that this is nearly 100% identical to the current softfork functionality.
If you used the drop-into-spv option during the bip66 fiasco you would have had exactly the same outcome as a non-upgraded node.
But soft forks are ugly and make the codebase complex!
If you look at the historical codebase, Satoshi themselves appears to have realised soft forks were possible after the first release, adding, for instance, ten NOP opcodes that we’ve since used for upgrades. In my own CHECKLOCKTIMEVERIFY making it a soft fork was an obvious decision – the hard fork version of it is a cosmetic change, precisely because it can use one of the NOP’s Satoshi added.
Not all softforks will result in ugly protocol design (apologies if I gave that impression). What I had in mind was p2sh and segwit which could have been designed much better by dropping the softfork requirement. In the case where the softfork just redefines an OP_NOP, my criticisms are restricted to those related to security and process.
But soft forks are undemocratic!
An interesting non-technical objection is that because nodes and miners who haven’t adopted the soft fork end up in the main chain anyway, this is a case of a majority undemocratically forcing a minority to adopt new policies. Of course, it’s your choice to use a protocol with this feature! Satoshi could have just as easily written Bitcoin to treat unknown block versions as invalid, but choose not too.
In today’s development environment, softforks can be rolled out with the approval of only a handful of core developers and a handful of mining pool operators. The grand total of people needed to sign off on a change right now is about a dozen. This is part of the reason why many of us feel development and governance are broken.
Remember back to the example above where Peter said your options in the face of a softfork are:
2) Take a security hit and continue
3) Turn off your node and stop using bitcoin.
With a hardfork you have a fourth option ― refuse to upgrade and continue on the current chain. If enough people threatened to do this (meaning there isn’t broad community consensus for the change), the miners will be effectively blocked from upgrading since they wont want to risk the coins they mine being unspendable. The hardfork process prevents the scenario where 10 people ram through a change over the objections of everyone else.
When Peter says, “Of course, it’s your choice to use a protocol with this feature!”, he’s basically suggesting that if you don’t like the possibility that the rules governing your money can be changed by a committee of 10 people then you don’t have to use Bitcoin!
I’m tempted to comment further but I’m pretty sure that statement speaks for itself.
More pragmatically, if 95%+ of the hashing power disagree with what you think the protocol should be, the remaining 5% chain probably isn’t secure anyway, so at minimum choosing to still use it should be an explicit choice (and you probably need a new PoW function).
This would be a much stronger argument if mining was not consolidated into the hands of just a few pool operators. If you want to ram through a change by softfork, all you need to do is convince these people to run your code:
The process of making protocol changes by hardfork is inherently more inclusive as those miners will have to consider what the rest of the community, where they will be spending their mined coins, wants. If 95% of miners upgrade in a hardfork, you will still need to cave in and upgrade to use Bitcoin, but at least you know the reason the miners upgraded is because nearly everyone else supported the change. The same cannot be said for softforks where the change could (in theory) be pushed through over the objections of the rest of the community.
Could softforks be made less objectionable?
I think they could but they would end up resembling the outcomes you’d get with a hardfork. Consider the following objections and how you would address them:
Involuntary security reduction
If softforks had a grace period similar to BIP101 your node could be made notify you (preferably by email or SMS as very few people read the logs) that you have two weeks to upgrade or else suffer a security downgrade. At least here people are give an explicit choice and the security downgrade doesn’t just hit them out of the blue. If they choose not to upgrade even after the warning then I think you could conclude they consented to the security downgrade.
Full node veto
As I said earlier, in a hardfork full nodes get an effective veto of protocol changes since refusal by super-majority to upgrade will block any change from going through (since miners wont want to risk mining coins they can’t spend). But with a softfork, miners can force a change on people since their nodes just follow the longest chain.
But this could be changed to give node operators the option to reject blocks with a version number they disapprove of. For example, if miners want to upgrade to version 5 blocks but nearly everyone objects, node operators could instruct their node to reject version 5 blocks and block the change. Just like with a hardfork.
The fact that today full node operators don’t have either of these options when a softforks rolls out is the source of most of my objections. But note that if these changes were made, softforks would just end up looking like hardforks anyway. Except with hardforks you still get a better protocol design in cases like segwit.