To understand transaction malleability, you should think of Bitcoin as a global ledger and transactions as bank checks.
A transaction identifier, called a transaction hash, is something like a fingerprint and is unique. Unlike a fingerprint, the transaction ID changes if the transaction it represents changes in any way.
Much like bank checks have written signatures, Bitcoin transactions have digital signatures. Much like real life signatures, digital signatures can vary slightly and yet still be valid.
Because the transaction identifier takes into account everything in the transaction, the transaction identifier changes if the digital signature changes. These subtle changes can happen only before a transaction is put into a block. Once a transaction is added to the blockchain, the transaction, including the signature becomes immutable.
The mutability of the signature can be problematic because someone can take a transaction and modify the signature slightly and make the transaction identifier change.
This would be the equivalent of replacing the signature on a check. The amount on the check would not change, but changing the signature would cause the check identifier to change.
Transaction identifiers are particularly important. New transactions refer to past transaction identifiers to prove that the signature on the check is correct.
How does transaction malleability affect the Lightning Network?
The Lightning Network is an off-chain micro-payment network. The Lightning Network works by creating a double-signed transaction. That is, we have a new check that requires both parties to sign for it to be valid. The check specifies how much is being sent from one party to another. As new micro-payments are made from one party to the other, the amount on the check is changed and both parties sign the result.
To start the Lightning Network channel, this double-signed check must be funded. In order that the double-signed check isn’t held by an uncooperative counterparty, the double-signed check is signed by both parties before the funding transaction is sent out to the network.
For the Lightning Network to work, we need the funding transaction to not be broadcast until the double-signed check is signed.
Because the double-signed check refers to the funding transaction’s identifier, if the funding transaction’s identifier is changed, the double-signed check will become invalid. This represents a risk to opening the Lightning Network channel. There are methods to make the Lightning Network work without this fixing transaction malleability, but Lightning Network is easier when transaction malleability is fixed.
How does Segregated Witness (SegWit) fix transaction malleability?
It turns out signatures are the only way in which transaction identifiers can be changed by an attacker. With SegWit, the transaction identifier no longer takes into account the signature. This means that even if the attacker changes the signature, the transaction identifier stays the same. Signatures are still checked, just not used in calculating the transaction identifier.
Fixing transaction malleability means that the Lightning Network can work smoothly.
