Canonical: https://solanashreds.xyz/docs/decode/python

# A Python decoder for research and alerting

Capture the stream with tcpdump, then turn the capture into a table: one row per transaction, decoded by solders and written to Parquet for pandas, DuckDB or Spark. Tested on two real mainnet blocks and cross-checked against a second library.

To decode Solana transactions in Python, capture the shreds with tcpdump, read the UDP payloads, join data shreds into block components, cut out each transaction's bytes, and parse them with solders, which handles legacy, v0 and v1. The script below does all of that and writes one Parquet row per transaction, ready for pandas.

## Why Python reads captures better than live feeds

A live feed never waits. If the program on the socket falls behind, the kernel buffer fills and new packets are dropped, and Python spends far more time per packet than a compiled decoder does. A capture removes that pressure: tcpdump writes the packets to disk at wire speed, and Python reads them back afterwards at whatever pace it manages, with nothing lost to slowness.

That suits the work Python is good at. Research, backtests and one-off questions start from a fixed set of blocks, and a capture plus a table is easy to rerun, share and load into a notebook. For a decoder that must keep up with the stream itself, use the [Rust decoder](https://solanashreds.xyz/docs/decode/rust), the [TypeScript decoder](https://solanashreds.xyz/docs/decode/typescript) or the [Go decoder](https://solanashreds.xyz/docs/decode/go) instead.

## Steps

1. Capture: `sudo tcpdump -ni any -w shreds.pcap udp port 8001`, with your own port, for as long as you need, then stop it with Ctrl-C.
2. Install the two dependencies, pinned in `requirements.txt` below.
3. Run `python3 decode_capture.py shreds.pcap 8001 transactions.parquet`.
4. Load the table: `pyarrow.parquet.read_table("transactions.parquet")`, or `.to_pandas()` on the result if pandas is installed.

Inside the script the work runs in order: read each frame and keep IPv4 UDP traffic to your port; parse the 88-byte data shred header; group shreds by slot until a run ending in the DATA_COMPLETE flag forms a block component; walk the entries in that component; measure each transaction and hand exactly its bytes to `VersionedTransaction.from_bytes`; then collect the rows and write Parquet.

## requirements.txt

```text
solders==0.29.0
pyarrow==26.0.1
```

## decode_capture.py

```python
"""Decode Solana transactions from a packet capture of raw shreds, and write them to a Parquet table.

    pip install solders==0.29.0 pyarrow==26.0.1
    sudo tcpdump -ni any -w shreds.pcap udp port 8001        # capture for a while, then Ctrl-C
    python3 decode_capture.py shreds.pcap 8001 transactions.parquet

Reads classic pcap files (tcpdump -w). Tested on Ethernet and Linux "any" (SLL2) captures; the SLL v1 and raw IP
framings are handled the same way but untested.
Uses data shreds only: a lost data shred costs the block component it belongs to.
"""
import struct
import sys
from collections import defaultdict

import pyarrow as pa
import pyarrow.parquet as pq
from solders.transaction import VersionedTransaction

DATA_COMPLETE = 0x40


# ---------- 1. packets out of the capture ----------
def udp_payloads(path, port):
    """Yield the UDP payloads sent to `port`, in capture order."""
    with open(path, "rb") as f:
        head = f.read(24)
        magic = head[:4]
        endian = "<" if magic in (b"\xd4\xc3\xb2\xa1", b"\x4d\x3c\xb2\xa1") else ">"
        linktype = struct.unpack(endian + "I", head[20:24])[0] & 0x0FFFFFFF
        offset = {1: 14, 113: 16, 276: 20, 101: 0, 228: 0}.get(linktype)
        if offset is None:
            raise SystemExit(f"unsupported link type {linktype}: capture with tcpdump -i any or on an Ethernet interface")
        while True:
            rec = f.read(16)
            if len(rec) < 16:
                return
            caplen = struct.unpack(endian + "IIII", rec)[2]
            frame = f.read(caplen)
            ip = frame[offset:]
            if len(ip) < 28 or ip[0] >> 4 != 4 or ip[9] != 17:      # IPv4 carrying UDP only
                continue
            udp = ip[(ip[0] & 0x0F) * 4:]
            if struct.unpack(">H", udp[2:4])[0] == port:
                yield udp[8:]


# ---------- 2. shred header ----------
def data_shred(buf):
    """(slot, index, flags, payload) for a Merkle data shred, else None."""
    if len(buf) < 88 or buf[64] & 0xF0 not in (0x90, 0xB0):
        return None
    slot, index = struct.unpack_from("<QI", buf, 65)
    flags, size = buf[85], struct.unpack_from("<H", buf, 86)[0]
    if size < 88 or size > len(buf):
        return None
    return slot, index, flags, bytes(buf[88:size])


# ---------- 3. slot assembly: a run of data shreds ending in DATA_COMPLETE is one block component ----------
class Assembler:
    def __init__(self):
        self.parts = defaultdict(dict)
        self.starts = defaultdict(lambda: {0})
        self.done = defaultdict(set)

    def push(self, slot, index, flags, payload):
        parts = self.parts[slot]
        if index in parts:
            return
        parts[index] = (flags, payload)
        if flags & DATA_COMPLETE:
            self.starts[slot].add(index + 1)
        for start in sorted(self.starts[slot]):
            if start in self.done[slot]:
                continue
            end = start
            while end in parts and not parts[end][0] & DATA_COMPLETE:
                end += 1
            if end not in parts:
                continue
            self.done[slot].add(start)
            yield b"".join(parts[i][1] for i in range(start, end + 1))


# ---------- 4. entry framing ----------
def shortvec(b, o):
    value = shift = 0
    while True:
        byte = b[o]
        o += 1
        value |= (byte & 0x7F) << shift
        if not byte & 0x80:
            return value, o
        shift += 7


def tx_length(b, start):
    """Bytes in the serialized transaction at `start`: legacy, v0, or v1 (version byte 129, signatures last)."""
    if b[start] == 129:
        ixs, addrs = b[start + 40], b[start + 41]
        words = bin(struct.unpack_from("<I", b, start + 4)[0]).count("1")
        heads = start + 42 + 32 * addrs + 4 * words
        o = heads + 4 * ixs
        for i in range(ixs):
            o += b[heads + 4 * i + 1] + struct.unpack_from("<H", b, heads + 4 * i + 2)[0]
        return o + 64 * b[start + 1] - start
    n, o = shortvec(b, start)
    o += 64 * n
    v0 = b[o] & 0x80
    o += 1 if v0 else 0
    n, o = shortvec(b, o + 3)
    o += 32 * n + 32
    k, o = shortvec(b, o)
    for _ in range(k):
        n, o = shortvec(b, o + 1)
        n, o = shortvec(b, o + n)
        o += n
    if v0:
        k, o = shortvec(b, o)
        for _ in range(k):
            n, o = shortvec(b, o + 32)
            n, o = shortvec(b, o + n)
            o += n
    return o - start


def transactions(component):
    """Exact bytes of each transaction in a block component, in order. A zero entry count marks a header or footer."""
    (entries,) = struct.unpack_from("<Q", component, 0)
    o = 8
    for _ in range(entries):
        o += 40                                             # num_hashes and the entry hash
        (count,) = struct.unpack_from("<Q", component, o)
        o += 8
        for _ in range(count):
            n = tx_length(component, o)
            yield component[o:o + n]
            o += n


# ---------- 5. decode with solders, 6. write Parquet ----------
def row(slot, raw):
    tx = VersionedTransaction.from_bytes(raw)
    msg = tx.message
    keys = [str(k) for k in msg.account_keys]
    version = "legacy" if type(msg).__name__ == "Message" else "1" if type(msg).__name__ == "MessageV1" else "0"
    return {
        "slot": slot,
        "signature": str(tx.signatures[0]),
        "version": version,
        "fee_payer": keys[0],
        "programs": sorted({keys[ix.program_id_index] for ix in msg.instructions}),
        "instructions": len(msg.instructions),
    }


def main(capture, port, out):
    asm, rows = Assembler(), []
    for payload in udp_payloads(capture, port):
        shred = data_shred(payload)
        if shred:
            for component in asm.push(*shred):
                rows.extend(row(shred[0], raw) for raw in transactions(component))
    pq.write_table(pa.Table.from_pylist(rows), out)
    print(f"{len(rows)} transactions written to {out}")
    for r in rows[:5]:
        print(r["slot"], r["signature"], r["version"])


if __name__ == "__main__":
    main(sys.argv[1], int(sys.argv[2]), sys.argv[3])
```

## The table it writes

| Column | Type | Meaning |
| --- | --- | --- |
| slot | integer | the slot the transaction's block belongs to |
| signature | string | the first signature, which is the transaction's ID |
| version | string | `legacy`, `0` or `1` |
| fee_payer | string | the first account key, which pays the fee |
| programs | list of strings | programs called by top-level instructions, sorted |
| instructions | integer | number of top-level instructions (can be 0 for some v1 transactions) |

Every column comes from bytes you received. Nothing here says whether a transaction succeeded; join with confirmed data from RPC if you need that.

## Fixture run

We wrote each of two mainnet blocks, slots 470,901,815 and 470,901,816 as shredded by Agave 4.4.1, into a standard pcap file twice: with Ethernet framing, as tcpdump writes on a single interface, and with Linux SLL2 framing, as `tcpdump -i any` writes. Traffic on another port was mixed in. These are generated captures in the real file format, not live recordings. Test: `python test_decode.py` on Python 3.11 with solders 0.29.0 and pyarrow 26.0.1.

| Check | Result |
| --- | --- |
| Transactions read back from Parquet, both framings | 269 of 269 and 512 of 512, in block order |
| Version and program list against @solana/kit, a separate decoder | identical on all 781 rows |
| Traffic to another port | ignored |
| Real tcpdump 4.99.4 captures of a loopback replay, `-i any` and `-i lo`, run by a second engineer | 269 of 269 and 512 of 512, in block order, signatures equal to getBlock |

## What this decoder leaves out

It keeps data shreds only and does not rebuild lost ones from coding shreds, so a packet missing from the capture costs every transaction in its block component. Captures taken under load also drop packets in the kernel, which tcpdump reports when it stops; check that count before you trust a dataset.

It reads classic pcap files, not pcapng, and IPv4 only. Account keys are the static keys: a v0 transaction that loads accounts from lookup tables shows only those it lists directly, and [resolving lookup tables](https://solanashreds.xyz/docs/decode/resolve-lookup-tables) fills in the rest. Inner instructions and execution results are never in shreds.

## Questions

### How do I decode Solana transactions in Python?

Use solders. Its `VersionedTransaction.from_bytes` parses legacy, v0 and v1 transactions; version 0.29.0 is the one we tested. The work around it, from packets to the exact bytes of each transaction, is the script on this page.

### Can I run this on a live socket instead?

You can feed the same functions from a socket, but on a busy stream Python will fall behind and the kernel will drop packets. Capture first, or use a compiled decoder for live work.

### Why do some rows have zero instructions?

Some v1 transactions on mainnet carry no instructions and only pay a fee. They are genuine and decode correctly.

## Next

[A research dataset from raw shreds](https://solanashreds.xyz/blog/research-datasets-from-shreds) covers what to keep, and [archiving raw shreds](https://solanashreds.xyz/compare/shred-archive-formats) compares ways to store captures for replay. [What is inside a Solana transaction](https://solanashreds.xyz/learn/solana-transaction-structure) explains the columns. To check a decoder of your own against known output, follow [testing against fixtures](https://solanashreds.xyz/docs/decode/test-fixtures). To capture your own stream, see [pricing](https://solanashreds.xyz/pricing).
