Canonical: https://solanashreds.xyz/docs/decode/test-fixtures

# Testing your decoder against fixtures and RPC

A decoder is right when it reproduces what the chain recorded. Capture a slot's shreds, decode them, and diff the result against RPC getBlock for the same slot: same signatures, same order, same accounts. This page gives the capture command and a tested diff script.

To test a Solana decoder, capture the shreds for one slot, decode them, then fetch the same slot with RPC `getBlock` and compare: every decoded signature must appear in the block, in block order, with the same static account keys. Missing transactions mean lost shreds; anything wrong or extra means a decoder bug.

## The three fixture types

| Fixture | What it is | Used to check |
| --- | --- | --- |
| Slot capture | every datagram your port received during one or more slots, as a pcap file | the whole path from packet to transaction |
| Expected output | `getBlock` for the same slots, saved as JSON | the truth the decoder is compared with |
| Program samples | individual transactions that call the programs you decode, with their metadata | instruction decoders, against what the program logged and moved |

Save all three next to your tests so CI can run without a network. Keep the expected output with `maxSupportedTransactionVersion: 1`; without it, any block containing a v1 transaction makes the request fail.

## Steps

1. Capture your feed's UDP port for a short while with tcpdump, as below.
2. Decode the capture with your decoder.
3. Fetch `getBlock` for every slot the capture contains, or read the saved copies.
4. Compare signatures in order and static keys per transaction.
5. Run the comparison in CI against the saved files, and occasionally against live RPC for a fresh capture.

## Capture

```bash
# Capture the UDP port your shreds arrive on (8001 here) into a pcap file. Stop with Ctrl-C after a few slots.
sudo tcpdump -i any -n -w capture.pcap udp port 8001
```

`-i any` works on any Linux host; the script reads that interface's link type, plain Ethernet, and loopback captures. Open the port in your firewall for UDP from any source port, because the sending port is not fixed.

## oracle_diff.py

```python file="oracle_diff.py"
"""Test a shred decoder against RPC truth: decode a packet capture, fetch getBlock for each slot it contains, and
compare signatures (in block order) and static account keys transaction by transaction.

    python3 oracle_diff.py capture.pcap [--port 8001] [--rpc URL | --blocks DIR]

--rpc    fetch getBlock live (default: the public mainnet endpoint; mind its rate limits)
--blocks read saved getBlock results (<slot>.json) instead, for repeatable tests in CI
Exit code 0 only when every slot matches.
"""
import argparse
import base64
import json
import struct
import sys
import time
import urllib.request

from shreds import SlotAssembler, parse_component, parse_shred
from tx import decode_tx


def udp_payloads(path: str, port: int | None):
    """Yield UDP payloads from a classic pcap file (what `tcpdump -w` writes)."""
    data = open(path, "rb").read()
    magic = data[:4]
    endian = "<" if magic in (b"\xd4\xc3\xb2\xa1", b"\x4d\x3c\xb2\xa1") else ">"
    linktype = struct.unpack(endian + "I", data[20:24])[0]
    link_len = {1: 14, 113: 16, 276: 20, 0: 4}.get(linktype)  # Ethernet, Linux cooked v1 and v2, BSD loopback
    if link_len is None:
        sys.exit(f"unsupported pcap link type {linktype}")
    o = 24
    while o + 16 <= len(data):
        incl = struct.unpack(endian + "I", data[o + 8:o + 12])[0]
        frame = data[o + 16:o + 16 + incl]
        o += 16 + incl
        ip = frame[link_len:]
        if len(ip) < 20 or ip[0] >> 4 != 4 or ip[9] != 17:  # IPv4 carrying UDP only
            continue
        udp = ip[(ip[0] & 0x0F) * 4:]
        if port is not None and struct.unpack(">H", udp[2:4])[0] != port:
            continue
        yield udp[8:]


def decode_capture(path: str, port: int | None) -> dict[int, list]:
    """slot -> decoded transactions, in the order their components complete."""
    asm, out = SlotAssembler(), {}
    for payload in udp_payloads(path, port):
        shred = parse_shred(payload)
        if shred is None:
            continue
        for run in asm.push(shred):
            c = parse_component(run)
            if c.kind == "entries":
                out.setdefault(shred.slot, []).extend(decode_tx(t) for e in c.entries for t in e.transactions)
    return out


def get_block(slot: int, rpc: str | None, blocks: str | None) -> dict:
    if blocks:
        return json.load(open(f"{blocks}/{slot}.json"))
    body = json.dumps({"jsonrpc": "2.0", "id": 1, "method": "getBlock", "params": [slot, {
        "encoding": "base64", "transactionDetails": "full", "rewards": False,
        "maxSupportedTransactionVersion": 1}]}).encode()  # without this, blocks with v1 transactions fail
    for attempt in range(6):
        req = urllib.request.Request(rpc, body, {"content-type": "application/json"})
        try:
            reply = json.load(urllib.request.urlopen(req, timeout=60))
        except urllib.error.HTTPError as e:
            if e.code == 429:
                time.sleep(2 ** attempt)
                continue
            raise
        if "error" in reply:
            sys.exit(f"getBlock {slot}: {reply['error']}")
        return reply["result"]
    sys.exit(f"getBlock {slot}: rate limited")


def compare(slot: int, decoded: list, block: dict) -> bool:
    truth = [decode_tx(base64.b64decode(t["transaction"][0])) for t in block["transactions"]]
    want = {t.signatures[0]: t for t in truth}
    got_sigs = [t.signatures[0] for t in decoded]
    missing = [s for s in want if s not in set(got_sigs)]
    foreign = [s for s in got_sigs if s not in want]
    wrong_keys = [t.signatures[0] for t in decoded if t.signatures[0] in want and t.static_keys != want[t.signatures[0]].static_keys]
    order_ok = got_sigs == [t.signatures[0] for t in truth if t.signatures[0] in set(got_sigs)]
    ok = not foreign and not wrong_keys and order_ok
    print(f"slot {slot}: decoded {len(got_sigs)}/{len(want)}, missing {len(missing)}, foreign {len(foreign)}, "
          f"wrong static keys {len(wrong_keys)}, order {'ok' if order_ok else 'WRONG'}")
    if missing and ok:
        print(f"  {len(missing)} missing: lost shreds, not a decoder bug (nothing decoded is wrong)")
    return ok and not missing


def main() -> None:
    p = argparse.ArgumentParser()
    p.add_argument("pcap")
    p.add_argument("--port", type=int)
    p.add_argument("--rpc", default="https://api.mainnet-beta.solana.com")
    p.add_argument("--blocks")
    a = p.parse_args()
    decoded = decode_capture(a.pcap, a.port)
    if not decoded:
        sys.exit("no shreds decoded from the capture")
    results = [compare(slot, txs, get_block(slot, a.rpc, a.blocks)) for slot, txs in sorted(decoded.items())]
    sys.exit(0 if all(results) else 1)


if __name__ == "__main__":
    main()
```

It needs Python 3.10 or later and nothing outside the standard library apart from `shreds.py` and `tx.py`, the parsers from the [Python decoder](https://solanashreds.xyz/docs/decode/python). Run `python3 oracle_diff.py capture.pcap --port 8001` for live RPC, or add `--blocks DIR` to use saved `getBlock` files.

## What a diff means

| Result | Meaning |
| --- | --- |
| everything matches | the decoder reproduced the slot exactly |
| missing transactions, nothing foreign, keys and order correct | shreds were lost before or during capture; whole block components drop out at once |
| foreign signatures | the decoder invented transactions, usually by reading one transaction's bytes with the wrong format |
| wrong static keys | a parsing bug in the account list, or the wrong format version |
| order wrong | runs or entries assembled out of order |

Lost shreds are a property of the capture, not the decoder. If you need a capture with nothing missing, decode it with a receiver that recovers from coding shreds, such as the [Rust decoder](https://solanashreds.xyz/docs/decode/rust).

## How this script was tested

We replayed two shred fixtures over UDP on loopback, captured them with tcpdump 4.99.4, and ran the script three ways. The fixtures were built locally with Agave's own Shredder from the real transactions of mainnet slots 470,901,815 and 470,901,816; no production traffic was captured.

| Case | Result |
| --- | --- |
| Full capture with `-i any` (Linux cooked frames) against saved `getBlock` | 269 of 269 and 512 of 512: nothing missing, foreign or wrong; exit 0 |
| The same replay captured with `-i lo` (Ethernet frames) | the same result |
| Full capture against live mainnet `getBlock` | the same result |
| Capture with 1 in 97 data shreds dropped | 128 of 269 and 128 of 512 decoded; the rest reported as lost shreds, with nothing foreign and no wrong keys; exit 1 |
| Truth tampered with: two transactions swapped in one slot, one key byte changed in the other | order WRONG and 1 wrong key reported; exit 1 |

## Keeping fixtures small

A capture of two slots came to about 2.7 MB, and saved `getBlock` results for the same slots to about 5 MB. A handful of slots is enough to cover all three transaction versions, lookup tables and the programs you decode. Choose them deliberately: a slot with v1 transactions, one with many v0 lookups, one with each program your decoder supports. Store large captures outside the repository and fetch them in CI.

## Old captures contain votes

Captures from before Alpenglow are full of vote transactions: votes left the blocks from slot 459,653,000, and current blocks contain none. A decoder tested only on old captures may handle votes perfectly and still fail on what blocks contain now, and a test that counts transactions will disagree wildly between old and new fixtures. Refresh fixtures after network upgrades, and date them. [Versioned transactions](https://solanashreds.xyz/learn/versioned-transactions) lists the formats a current fixture should include.

## What this test cannot prove

Agreement with `getBlock` proves the decoder reproduced the transactions; it says nothing about instruction decoding, which needs the program checks described in [regression tests for decoders](https://solanashreds.xyz/blog/regression-tests-for-decoders). It also cannot test what the capture never contained, and live RPC results depend on the node still holding the slot. Malformed input is a separate concern: [your decoder will meet garbage](https://solanashreds.xyz/blog/your-decoder-will-meet-garbage) covers fuzzing.

## Next

Settle what the decoded transactions did with [reconciliation](https://solanashreds.xyz/docs/pipeline/reconcile), or keep captures as datasets with [shred archive formats](https://solanashreds.xyz/compare/shred-archive-formats). For a feed to capture from, [see pricing](https://solanashreds.xyz/pricing).
