BIP62 - Dealing with Malleability¶

BIP62 (Bitcoin Improvement Proposal 62) is one of the Bitcoin improvement proposals, put forward by Pieter Wuille in 2014. It aimed to reduce or eliminate the transaction malleability problem by defining a series of rules. Although this proposal was ultimately never activated as a consensus rule, its analysis and solutions had a profound impact on the development of the Bitcoin protocol and laid the theoretical foundation for subsequent major upgrades such as Segregated Witness (BIP141).

Core Concept¶

Transaction Malleability refers to the issue where a transaction's ID (txid) can be modified before the transaction is confirmed. Although malleability attacks do not result in stolen funds, they can change the transaction's unique identifier, breaking subsequent transactions that depend on that ID. This creates serious problems for smart contracts, the Lightning Network, and other advanced applications.

BIP62 identified seven major sources of malleability and proposed corresponding restriction rules for each. These rules, by standardizing the transaction format, eliminate the ability of third parties to modify transaction IDs, keeping transactions stable from broadcast until confirmation.

Problem Background¶

What is Transaction Malleability¶

The transaction ID is the double SHA256 hash of the transaction content:

txid = SHA256(SHA256(serialized transaction data))

Malleability Problem: - Certain parts of a transaction can be modified without affecting its validity - The modified transaction is still valid, but the txid changes - Only one of the original and modified transactions can be confirmed - It is impossible to predict which version will be confirmed

Dangers of Malleability¶

1. Breaking Transaction Chains:

Scenario: Alice creates transaction A, Bob creates transaction B based on A

Step 1: Alice broadcasts transaction A (txid = 0x123...)
Step 2: Bob creates transaction B, referencing input 0x123...
Step 3: A third party modifies transaction A's signature, producing A' (txid = 0x456...)
Step 4: Transaction A' is confirmed instead of A
Step 5: Transaction B is invalidated because 0x123... does not exist

Result: Bob's transaction can never be confirmed

2. Fraudulent Claims of Non-Receipt:

Scenario: Alice withdraws from an exchange

Step 1: Exchange creates withdrawal transaction T1 (txid = 0xAAA...)
Step 2: Alice modifies T1 to T1' (txid = 0xBBB...)
Step 3: T1' is confirmed; Alice actually receives the funds
Step 4: Exchange queries 0xAAA... and finds it unconfirmed
Step 5: Alice claims she did not receive the funds and requests a re-withdrawal
Step 6: Exchange may be deceived and pays again

Result: Alice may receive double withdrawal (if the exchange system is flawed)

3. Hindering Layer 2 Development: - The Lightning Network needs to construct chains of unconfirmed transactions - Smart contracts need to know transaction IDs in advance - Atomic swaps need reliable transaction references - Malleability makes these applications unsafe or infeasible

Historical Events¶

2014 Mt. Gox Incident: - Mt. Gox, the largest Bitcoin exchange at the time, went bankrupt - Transaction malleability attacks were partly blamed - Users modified the txid of withdrawal transactions - Claimed non-receipt and obtained duplicate withdrawals - While malleability was not the sole cause of Mt. Gox's collapse, it exposed the severity of the problem

Sources of Malleability¶

BIP62 identified seven major sources of malleability:

1. Signature Malleability¶

Problem: ECDSA signatures are inherently malleable. For a signature (r, s), (r, -s mod n) is also a valid signature.

Example:

Original signature: (r, s)
Modified signature: (r, n - s)  where n is the order of the elliptic curve

Both signatures pass verification but produce different txids

BIP62 Rule 1: Require signatures to use the low S value:

if (s > n/2) {
    s = n - s
}

Implementation Status: - Partially addressed through BIP66 (strict DER signatures) - Became a standard transaction rule after Bitcoin Core 0.11.1 - SegWit in 2017 completely eliminated signature malleability

2. Non-DER Encoded Signatures¶

Problem: DER (Distinguished Encoding Rules) encoding allows multiple valid representations: - Length fields can have extra zeros - Integers can have leading zeros - Signatures can include extra padding

Example:

Standard DER:    30 44 02 20 [r] 02 20 [s]
Non-standard DER: 30 45 02 21 00 [r] 02 20 [s]  <- extra 00

BIP62 Rule 2: Enforce strict DER encoding.

Implementation Status: - BIP66 activated in July 2015, enforcing strict DER - Became a consensus rule, fully resolving this issue

3. Extra Data in Scripts¶

Problem: Some script operations allow adding extra data without affecting execution:

Example:

Original script: <sig> <pubkey>
Modified script: <sig> OP_NOP <pubkey>

Or

Original script: OP_0
Modified script: OP_0 OP_0 OP_DROP

BIP62 Rule 3: Prohibit adding meaningless operations in scripts.

Implementation Status: - Partially implemented through standard transaction rules - SegWit fully resolved this by removing signature data

4. scriptPubKey Type Malleability¶

Problem: Certain output script types allow multiple representations:

Example:

Standard P2PK:    <pubkey> OP_CHECKSIG
Alternative form: <pubkey> OP_NOP OP_CHECKSIG

Or

OP_0 can be written as OP_FALSE
OP_1 can be written as OP_TRUE

BIP62 Rule 4: Standardize scriptPubKey formats.

Implementation Status: - Partially implemented via IsStandard() checks - Restricted propagation of non-standard scripts

5. scriptSig Push Operations¶

Problem: Pushing data to the stack can use different opcodes:

Example:

Pushing 75 bytes of data:
Method 1: OP_PUSHDATA1 0x4b [75 bytes of data]
Method 2: 0x4b [75 bytes of data]  <- minimal encoding

Pushing 0:
Method 1: OP_0
Method 2: OP_PUSHDATA1 0x00
Method 3: 0x00  <- empty byte string

BIP62 Rule 5: Require minimal push operations: - 0-75 bytes: direct push - 76-255 bytes: OP_PUSHDATA1 - 256-65535 bytes: OP_PUSHDATA2 - 65536+ bytes: OP_PUSHDATA4

Implementation Status: - Became a standard rule after Bitcoin Core 0.11.1 - Did not become a consensus rule (BIP62 was not activated)

6. Spurious Push Operations¶

Problem: scriptSig can contain data that is never used:

Example:

P2SH redeem script: 2 <pubkey1> <pubkey2> 2 OP_CHECKMULTISIG

Standard scriptSig: OP_0 <sig1> <sig2> <redeem script>
Modified scriptSig: OP_0 <sig1> <sig2> <extra data> OP_DROP <redeem script>

BIP62 Rule 6: Prohibit unused push operations in scriptSig.

Implementation Status: - Difficult to enforce without breaking compatibility - SegWit resolved this through the witness data structure

7. OP_CHECKMULTISIG Dummy Stack Element¶

Problem: Due to a historical bug, OP_CHECKMULTISIG pops one extra stack element:

Example:

2-of-3 multisig:
Normal: OP_0 <sig1> <sig2> | 2 <pk1> <pk2> <pk3> 3 OP_CHECKMULTISIG

Malleability modification:
OP_0 can be replaced with OP_1, OP_1NEGATE, or any data
as long as the element is popped

BIP62 Rule 7: Require the dummy stack element of OP_CHECKMULTISIG to be OP_0.

Implementation Status: - Standard rule since Bitcoin Core 0.10+ - BIP147 (activated 2017) made it a consensus rule

Technical Details¶

Strict DER Signatures (BIP66)¶

Strict rules for DER encoding:

Signature format:
0x30 [total-length] 0x02 [R-length] [R] 0x02 [S-length] [S] [sighash-type]

Strict requirements:
1. Total length must equal the actual length
2. R and S must be minimal-length encoded (no leading zeros, unless the high bit is 1)
3. R and S must be positive numbers
4. S must be <= n/2 (low S value)
5. sighash type must be defined (0x01, 0x02, 0x03, 0x81, etc.)

Low S Value Verification¶

def verify_low_s(signature):
    """Verify signature uses low S value"""
    r, s = decode_signature(signature)

    # Order of the secp256k1 curve
    n = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141

    # Check s <= n/2
    if s > n // 2:
        return False

    return True

Minimal Push Verification¶

def verify_minimal_push(opcode, data):
    """Verify minimal push operation is used"""
    data_len = len(data)

    if data_len == 0:
        return opcode == OP_0
    elif data_len == 1:
        if 1 <= data[0] <= 16:
            return opcode == OP_1 + (data[0] - 1)
        elif data[0] == 0x81:
            return opcode == OP_1NEGATE
        else:
            return opcode == 0x01
    elif data_len <= 75:
        return opcode == data_len
    elif data_len <= 255:
        return opcode == OP_PUSHDATA1
    elif data_len <= 65535:
        return opcode == OP_PUSHDATA2
    else:
        return opcode == OP_PUSHDATA4

The Fate of BIP62¶

Proposal Status¶

BIP62 underwent lengthy discussion after being proposed:

2014: Proposal published, identifying seven sources of malleability

2015: Some rules were adopted - BIP66 (strict DER) activated, addressing Rule 2 - Low S value became a standard rule

2016-2017: Plans for BIP62 as a soft fork were shelved - Technical challenges in full implementation were discovered - Some rules were difficult to enforce - The decision was made to fundamentally solve the problem through SegWit

Final Result: - BIP62 was never activated as a consensus rule - Marked as "Withdrawn" - But its analysis and some of its rules were still adopted

Why It Was Not Activated¶

Technical Challenges: - Rules 5 and 6 were difficult to enforce without breaking compatibility - Certain edge cases were complex to handle - Could affect the validity of existing scripts

A Better Solution: - SegWit (BIP141) provided a more thorough solution - By separating witness data, it completely eliminated third-party malleability - SegWit also brought other benefits (capacity increase, signature optimization, etc.)

Community Consensus: - In 2016, the community decided to focus on SegWit - SegWit could solve multiple problems at once - Avoided the complexity of multiple soft forks

Relationship with Other BIPs¶

BIP66 (Strict DER Signatures)¶

BIP66 implemented Rule 2 of BIP62: - Activated in July 2015 - Forced all signatures to use strict DER encoding - Eliminated DER encoding malleability

BIP141 (Segregated Witness)¶

SegWit thoroughly solved the malleability problem:

Before SegWit:
txid = SHA256(SHA256(version + inputs + outputs + locktime))
Inputs contain signature data; signatures can be modified -> txid is mutable

After SegWit:
txid = SHA256(SHA256(version + inputs (no signatures) + outputs + locktime))
Signatures are in witness data, do not affect txid -> txid is immutable

Complete elimination of third-party malleability: - Signature data no longer factors into txid - Third parties cannot modify txid - Only the transaction creator can modify the txid by changing the input script (which requires the private key)

BIP147 (OP_CHECKMULTISIG Dummy Element)¶

BIP147 implemented Rule 7 of BIP62: - Activated in August 2017, simultaneously with SegWit - Forced the OP_CHECKMULTISIG dummy element to be OP_0 - Further reduced malleability vectors

BIP143 (SegWit Signature Hash)¶

BIP143 defined a new signature algorithm for SegWit transactions: - Commits to input amounts during signing - Prevents certain types of malleability - Improves hardware wallet security

Implemented Rules¶

Although BIP62 as a whole was not activated, some rules were implemented through other means:

Standard Transaction Rules¶

Bitcoin Core implements the following rules as IsStandard() checks:

Low S Value (Rule 1): - Enforced since version 0.11.1 - Non-standard transactions are not relayed - But miners can still include them in blocks

Minimal Push (Rule 5): - Enforced since version 0.11.1 - Improved consistency of standard transactions

OP_CHECKMULTISIG Dummy Element (Rule 7): - Standard rule since version 0.10.0 - Upgraded to consensus rule via BIP147

Consensus Rules¶

BIP66 (Rule 2): - Strict DER encoding - Activated July 2015

BIP147 (Rule 7): - OP_CHECKMULTISIG dummy element must be OP_0 - Activated August 2017

Practical Impact¶

Impact on Users¶

Exchanges and Wallets: - Need to implement robust txid tracking mechanisms - Cannot rely solely on txid to determine whether a transaction is confirmed - Should verify inputs and outputs, not just the txid

Best Practices:

Poor approach:
if (txid == expected_txid) {
    // Assume transaction confirmed
}

Correct approach:
if (transaction.inputs == expected_inputs &&
    transaction.outputs == expected_outputs &&
    transaction.confirmations >= 6) {
    // Transaction confirmed and content correct
}

Impact on Developers¶

Unconfirmed Transaction Chains: - Before SegWit: Unsafe, should be avoided - After SegWit: Safe, making Lightning Network and similar applications possible

Smart Contracts: - Before SegWit: Contracts depending on txid were unsafe - After SegWit: Parent transactions can be safely referenced

Impact on the Ecosystem¶

Lightning Network: - Requires constructing chains of unconfirmed transactions - Malleability fix was a prerequisite - After SegWit activation, the Lightning Network developed rapidly

Atomic Swaps: - HTLCs require reliable txids - Malleability fix made cross-chain atomic swaps possible

Security Considerations¶

Developer Guidelines¶

1. Do Not Rely Solely on txid:

# Incorrect example
def check_payment(expected_txid):
    tx = blockchain.get_transaction(expected_txid)
    return tx.confirmed

# Correct example
def check_payment(expected_address, expected_amount):
    txs = blockchain.get_transactions_to(expected_address)
    for tx in txs:
        if tx.amount == expected_amount and tx.confirmations >= 6:
            return True
    return False

2. Use SegWit Addresses: - Prefer native SegWit (bc1q...) - Or nested SegWit (starting with 3...) - Completely avoids third-party malleability

3. Monitor All Transaction Versions: - Track all transactions spending the same inputs - Identify which version is confirmed - Update internal records promptly

Exchange Security¶

Withdrawal Systems:

1. Create withdrawal transaction T
2. Record the transaction's inputs and outputs, not just the txid
3. Broadcast the transaction
4. Monitor the blockchain for transactions matching the inputs/outputs
5. Upon confirmation, regardless of whether the txid matches, mark as complete
6. Do not respond to "txid not confirmed" complaints (verify inputs/outputs)

Double-Spend Detection: - Monitor all transactions spending the same UTXO - Distinguish between malleability modifications and genuine double-spends - Treat differently and take appropriate measures

Comparison: BIP62 vs SegWit¶

Historical Significance¶

BIP62's Contributions¶

Despite not being activated, BIP62 remains important:

Theoretical Foundation: - Systematically analyzed the malleability problem - Identified all major sources of malleability - Provided a framework for subsequent solutions

Partial Implementation: - Low S value became a standard rule - Strict DER implemented through BIP66 - OP_CHECKMULTISIG fix implemented through BIP147

Driving SegWit: - The challenges of BIP62 prompted the community to seek a better solution - SegWit emerged as a comprehensive solution - Solved malleability and other problems in one go

Lessons Learned¶

1. Complex Problems Need Comprehensive Solutions: - Patching malleability sources one by one was inefficient - SegWit solved the problem once and for all through an architectural change - Avoided the risk of multiple soft forks

2. Standard Rules vs. Consensus Rules: - Standard rules can iterate quickly - Consensus rules require caution and thorough testing - Both working together can achieve gradual improvement

3. Importance of Community Coordination: - Technical solutions require community consensus - Trade-offs between multiple small upgrades vs. one large upgrade - Thorough discussion prevented wrong directions

Summary¶

BIP62 is an important proposal in Bitcoin history. Although it was ultimately not activated, its influence has been profound:

Core Contributions: - Systematic analysis: First comprehensive identification of seven malleability sources - Theoretical foundation: Provided a framework for solving the malleability problem - Partial implementation: Partially addressed the problem through standard rules and other BIPs - Driving innovation: Prompted the community to develop the superior SegWit solution

Practical Value: - Strict DER (BIP66) eliminated signature encoding malleability - Low S value rule reduced signature malleability - OP_CHECKMULTISIG fix (BIP147) eliminated multisig malleability - Paved the way for SegWit's design and activation

Historical Significance: - Demonstrated the Bitcoin community's technical depth and iterative capability - Proved that sometimes a "better solution" is worth waiting for - The evolution from BIP62 to SegWit is a model of Bitcoin's gradual improvement - Cleared obstacles for revolutionary technologies like the Lightning Network

Final Outcome: Although BIP62 was never activated as an independent soft fork, the problems it identified have been thoroughly resolved through SegWit (BIP141). SegWit not only eliminated third-party malleability but also brought additional benefits such as capacity increase and signature optimization. In a sense, BIP62's "failure" was actually a success in the Bitcoin protocol evolution process -- through thorough discussion and technical exploration, the community found a superior solution.

Today, with widespread SegWit adoption, transaction malleability is no longer a major issue for Bitcoin. Advanced applications that depend on stable txids -- such as the Lightning Network, smart contracts, and atomic swaps -- are flourishing. BIP62's analysis and exploration, and the process of being ultimately superseded by SegWit, perfectly illustrate Bitcoin's evolution philosophy as an open-source protocol: continuous experimentation, thorough discussion, selecting the best approach, and ongoing improvement.