Canonical: https://solanashreds.xyz/docs/decode/rust

# A Rust decoder: shreds in, transactions out

A complete Rust receiver and decoder: it reads shreds from a UDP port, rebuilds lost data shreds from coding shreds, and prints every transaction as a JSON line. Every line of code below was tested against Agave's own shredder output.

A Solana transaction decoder in Rust can rest on three main crates: `solana-entry`, `solana-transaction` and a Reed-Solomon crate. It receives shreds over UDP, rebuilds lost data shreds, deserializes each block component and prints every legacy, v0 or v1 transaction as a JSON line with slot, signature, version, static keys and program IDs. No RocksDB, no validator state.

## Crates

| Crate | Version | Repository | What it does here |
| --- | --- | --- | --- |
| `solana-entry` | 4.4.1, feature `agave-unstable-api` | anza-xyz/agave | `BlockComponent`, `Entry` and their deserialization |
| `solana-transaction` | 5.1.0 | anza-xyz/solana-sdk | `VersionedTransaction` for legacy, v0 and v1 |
| `wincode` | 0.6.3 | crates.io | the serialization format block components use |
| `reed-solomon-erasure` | 6.0.0 | crates.io | rebuilds lost data shreds, as Agave does |
| `serde_json` | 1.0.154 | crates.io | prints the JSON lines |

Built with Rust 1.98.1, the toolchain Agave 4.4.1 pins. `solana-entry` compiles to an empty crate unless you enable `agave-unstable-api`.

## Why not solana-ledger

`solana-ledger` holds Agave's shred types and the `Shredder`, so it looks like the obvious dependency. Two problems make it a poor fit for a receiver. From crates.io it only compiles with the forks Agave lists under `[patch.crates-io]` in its own `Cargo.toml` (rocksdb, librocksdb-sys, crossbeam-epoch and blstrs at fixed revisions), and it brings RocksDB with it. And its public recovery entry point expects a `Bank`, which only a validator has.

The shred header is small and fixed, so this decoder reads it by hand, uses `reed-solomon-erasure` for recovery the same way Agave does, and leaves the block format to `solana-entry`. The [shred parsing libraries comparison](https://solanashreds.xyz/compare/shred-parsing-libraries) weighs the alternatives.

## Steps

1. Receive each UDP datagram and read the variant byte at offset 64: `0x9_` and `0xB_` are data shreds, `0x6_` and `0x7_` coding shreds, anything else is dropped.
2. Read the slot, index and FEC set index from the common header, and for data shreds the flags and `size` field.
3. File each shred under its slot and its erasure set, and skip an index you already have.
4. When a set holds at least as many shreds as it has data shreds, rebuild the missing data shreds with Reed-Solomon.
5. Once a run of data shreds reaches one flagged `DATA_COMPLETE`, glue their payloads together and hand the bytes to `solana-entry`.
6. For an entry batch, print one JSON line per transaction; markers (block header and footer) carry no transactions and are skipped.

## Cargo.toml

```toml file="Cargo.toml"
[package]
name = "shred-receiver"
version = "0.1.0"
edition = "2024"
publish = false

# A receiver that does NOT need solana-ledger (no RocksDB, no Bank): headers are read by hand,
# lost data shreds are rebuilt from coding shreds with reed-solomon-erasure (the crate Agave uses),
# and block components are deserialized with solana-entry.
[dependencies]
solana-entry = { version = "=4.4.1", features = ["agave-unstable-api"] }
solana-transaction = "=5.1.0"
wincode = "=0.6.3"
reed-solomon-erasure = "=6.0.0"
serde_json = "=1.0.154"
```

## src/main.rs

```rust file="src/main.rs"
//! shred-receiver: UDP shreds -> (recover) -> block components -> transactions, printed as JSON lines.
//!   shred-receiver <udp-port>        or   shred-receiver --file <fixture.shreds>
//! Offsets and sizes: Agave v4.4.1 ledger/src/shred.rs and shred/merkle.rs (SOLANASHREDS-TECH-FACTS-E4 P-01 to P-08).
use {
    reed_solomon_erasure::galois_8::ReedSolomon,
    solana_entry::block_component::BlockComponent,
    std::collections::{BTreeMap, HashMap, HashSet},
};

const SIG: usize = 64;
const DATA_HEADERS: usize = 88;
const CODE_HEADERS: usize = 89;
const DATA_PAYLOAD: usize = 1203;
const CODE_PAYLOAD: usize = 1228;
const DATA_COMPLETE: u8 = 0x40;

struct Header {
    slot: u64,
    index: u32,
    fec: u32,
    is_data: bool,
    proof: usize,
    resigned: bool,
}

fn header(p: &[u8]) -> Option<Header> {
    let v = *p.get(64)?;
    let (is_data, resigned) = match v & 0xf0 {
        0x90 => (true, false),
        0xb0 => (true, true),
        0x60 => (false, false),
        0x70 => (false, true),
        _ => return None,
    };
    let u32_at = |o: usize| u32::from_le_bytes(p[o..o + 4].try_into().unwrap());
    if p.len() < if is_data { DATA_PAYLOAD } else { CODE_PAYLOAD } {
        return None;
    }
    Some(Header {
        slot: u64::from_le_bytes(p[65..73].try_into().unwrap()),
        index: u32_at(73),
        fec: u32_at(79),
        is_data,
        proof: (v & 0x0f) as usize,
        resigned,
    })
}

/// Bytes of a shred that are erasure coded: after the signature for data shreds, after the headers for
/// coding shreds, up to the chained Merkle root (always present now), proof and optional retransmitter signature.
fn shard_range(h: &Header) -> std::ops::Range<usize> {
    let tail = 32 + 20 * h.proof + if h.resigned { 64 } else { 0 };
    if h.is_data {
        SIG..DATA_PAYLOAD - tail
    } else {
        CODE_HEADERS..CODE_PAYLOAD - tail
    }
}

#[derive(Default)]
struct FecSet {
    data: BTreeMap<u32, Vec<u8>>,
    code: HashMap<u16, Vec<u8>>,
    num_data: u16,
    num_code: u16,
    sig: Vec<u8>,
    data_hdr_tail: Option<(usize, bool)>,
}

#[derive(Default)]
struct Slot {
    data: BTreeMap<u32, Vec<u8>>,
    fec: HashMap<u32, FecSet>,
    starts: HashSet<u32>,
    emitted: HashSet<u32>,
}

#[derive(Default)]
struct Assembler {
    slots: HashMap<u64, Slot>,
}

impl Assembler {
    /// Returns completed block components (raw bytes) ready to deserialize.
    fn push(&mut self, p: &[u8]) -> Vec<(u64, u32, Vec<u8>)> {
        let Some(h) = header(p) else { return vec![] };
        let st = self.slots.entry(h.slot).or_insert_with(|| Slot {
            starts: HashSet::from([0]),
            ..Default::default()
        });
        let set = st.fec.entry(h.fec).or_default();
        if set.sig.is_empty() {
            set.sig = p[..SIG].to_vec();
        }
        if h.is_data {
            if st.data.contains_key(&h.index) {
                return vec![];
            }
            set.data.insert(h.index, p[..DATA_PAYLOAD].to_vec());
            set.data_hdr_tail = Some((h.proof, h.resigned));
            Self::add_data(st, h.index, p[..DATA_PAYLOAD].to_vec());
        } else {
            let pos = u16::from_le_bytes(p[87..89].try_into().unwrap());
            set.num_data = u16::from_le_bytes(p[83..85].try_into().unwrap());
            set.num_code = u16::from_le_bytes(p[85..87].try_into().unwrap());
            set.code
                .entry(pos)
                .or_insert_with(|| p[..CODE_PAYLOAD].to_vec());
        }
        Self::try_recover(st, h.fec);
        Self::emit(h.slot, st)
    }

    fn add_data(st: &mut Slot, index: u32, payload: Vec<u8>) {
        if payload[85] & DATA_COMPLETE != 0 {
            st.starts.insert(index + 1);
        }
        st.data.insert(index, payload);
    }

    /// Rebuild missing data shreds once a FEC set holds at least num_data shreds in total.
    fn try_recover(st: &mut Slot, fec: u32) {
        let Some(set) = st.fec.get(&fec) else { return };
        let (nd, nc) = (set.num_data as usize, set.num_code as usize);
        if nd == 0 || set.data.len() >= nd || set.data.len() + set.code.len() < nd {
            return;
        }
        let Some((proof, resigned)) = set.data_hdr_tail.or_else(|| {
            set.code
                .values()
                .next()
                .map(|c| ((c[64] & 0x0f) as usize, c[64] & 0xf0 == 0x70))
        }) else {
            return;
        };
        let dh = Header {
            slot: 0,
            index: 0,
            fec: 0,
            is_data: true,
            proof,
            resigned,
        };
        let ch = Header {
            is_data: false,
            ..dh
        };
        let mut shards: Vec<Option<Vec<u8>>> = (0..nd)
            .map(|i| {
                set.data
                    .get(&(fec + i as u32))
                    .map(|p| p[shard_range(&dh)].to_vec())
            })
            .collect();
        shards.extend((0..nc).map(|i| {
            set.code
                .get(&(i as u16))
                .map(|p| p[shard_range(&ch)].to_vec())
        }));
        let Ok(rs) = ReedSolomon::new(nd, nc) else {
            return;
        };
        if rs.reconstruct_data(&mut shards).is_err() {
            return;
        }
        let sig = set.sig.clone();
        for i in 0..nd {
            let index = fec + i as u32;
            if st.data.contains_key(&index) {
                continue;
            }
            // Signature + recovered shard = the data shred up to its Merkle proof; headers and data are inside.
            let mut payload = sig.clone();
            payload.extend_from_slice(shards[i].as_ref().unwrap());
            payload.resize(DATA_PAYLOAD, 0);
            Self::add_data(st, index, payload.clone());
            st.fec.get_mut(&fec).unwrap().data.insert(index, payload);
        }
    }

    fn emit(slot: u64, st: &mut Slot) -> Vec<(u64, u32, Vec<u8>)> {
        let mut out = vec![];
        let mut starts: Vec<u32> = st.starts.iter().copied().collect();
        starts.sort_unstable(); // index order, so output order is stable
        for start in starts {
            if st.emitted.contains(&start) {
                continue;
            }
            let mut bytes = vec![];
            let mut i = start;
            loop {
                let Some(p) = st.data.get(&i) else { break };
                let size = u16::from_le_bytes(p[86..88].try_into().unwrap()) as usize;
                bytes.extend_from_slice(&p[DATA_HEADERS..size]);
                if p[85] & DATA_COMPLETE != 0 {
                    out.push((slot, start, std::mem::take(&mut bytes)));
                    st.emitted.insert(start);
                    break;
                }
                i += 1;
            }
        }
        out
    }
}

/// Prints one JSON line per transaction. `component` is the index of the first data shred of the block component
/// the transaction came from and `position` its place inside that component: together they order transactions within
/// a slot even when components complete out of order.
fn print_component(slot: u64, component: u32, bytes: &[u8]) {
    match wincode::deserialize::<BlockComponent>(bytes) {
        Ok(BlockComponent::EntryBatch(entries)) => {
            let mut position = 0u32;
            for e in entries {
                for tx in e.transactions {
                    let version = match tx.version() {
                        solana_transaction::versioned::TransactionVersion::Legacy(_) => {
                            "legacy".to_string()
                        }
                        solana_transaction::versioned::TransactionVersion::Number(n) => {
                            n.to_string()
                        }
                    };
                    let keys = tx.message.static_account_keys();
                    // Program ids are always static keys (a v0 message may not load them from lookup tables).
                    let mut programs: Vec<String> = tx
                        .message
                        .instructions()
                        .iter()
                        .map(|ix| keys[ix.program_id_index as usize].to_string())
                        .collect();
                    programs.dedup(); // collapses repeats that sit next to each other
                    println!(
                        "{}",
                        serde_json::json!({"slot": slot, "component": component, "position": position,
                    "signature": tx.signatures[0].to_string(), "version": version,
                    "static_keys": keys.iter().map(|k| k.to_string()).collect::<Vec<_>>(), "program_ids": programs})
                    );
                    position += 1;
                }
            }
        }
        Ok(BlockComponent::BlockMarker(_)) => {}
        Err(e) => eprintln!("slot {slot}: undecodable component ({e})"),
    }
}

fn main() {
    let args: Vec<String> = std::env::args().collect();
    let mut asm = Assembler::default();
    if args.get(1).map(String::as_str) == Some("--file") {
        let raw = std::fs::read(&args[2]).unwrap();
        let drop_every: usize = args.get(3).map(|s| s.parse().unwrap()).unwrap_or(0); // test: drop every Nth data shred
        let (mut o, mut k) = (0, 0);
        while o < raw.len() {
            let n = u16::from_le_bytes([raw[o], raw[o + 1]]) as usize;
            o += 2;
            let p = &raw[o..o + n];
            o += n;
            if drop_every > 0 && matches!(p[64] & 0xf0, 0x90 | 0xb0) {
                k += 1;
                if k % drop_every == 0 {
                    continue;
                }
            }
            for (slot, start, c) in asm.push(p) {
                print_component(slot, start, &c);
            }
        }
        return;
    }
    let sock = std::net::UdpSocket::bind(("0.0.0.0", args[1].parse::<u16>().unwrap())).unwrap();
    let mut buf = [0u8; 1280];
    loop {
        let Ok(n) = sock.recv(&mut buf) else { continue };
        for (slot, start, c) in asm.push(&buf[..n]) {
            print_component(slot, start, &c);
        }
    }
}
```

## Running it and what it prints

Build it with `cargo build --release`, allow inbound UDP to your port from any source port (the sending port is not fixed), then run `shred-receiver 8001` with the port your feed is delivered to.

The receiver ignores every datagram that is not a shred, and that includes the verification packet sent while your destination is being activated. During activation, read that packet with a plain UDP listener, for example `nc -ul 8001`, reply with its token, and start the decoder once delivery is on. [Checking that shreds reach your server](https://solanashreds.xyz/docs/start/verify-delivery) walks through it. The `--file` mode reads a saved capture in the same length-prefixed format the tests use.

Each transaction becomes one line. This is the first line from the fixture for slot 470,901,815, with the key list shortened here:

```json
{"component":32,"position":0,"program_ids":["Archer8kgiavM61GyusMzaaS2ft5sALtNsD1HxkUPMhy"],"signature":"36rFNmu3EShgKrPSipWyK6A8Yn58jWuHYrByCDDJCFVKzLCMiAGu4mFD4hw5zVKABJnenQSYmhbTWUhJfPsUKiuy","slot":470901815,"static_keys":["4M1ViRmsMu2vHBSsWK4b2PMKjQihwzXv78Luy3si5Moi","kmBG7GCnHbnhNkmb1aVPrsbebduhBaRqBqQJpGinmtE","..."],"version":"1"}
```

`component` names the block component a transaction came from by the shred index it starts at, and `position` counts transactions within that component. Sorting by the pair restores block order even when components finish out of order. `version` is `legacy`, `0` or `1`; the [v1 format](https://solanashreds.xyz/learn/transaction-v1) is the one that keeps its signatures at the end, and `solana-transaction` handles that for you.

## How this code was tested

The fixtures are two consecutive mainnet blocks, slots 470,901,815 and 470,901,816, fetched with `getBlock` and shredded locally by Agave 4.4.1's own `Shredder`. The transactions and their order are real. Entry grouping, entry hashes and the signing key are synthetic, because RPC does not expose entries and the real leader key is not ours to use.

| Test | Result |
| --- | --- |
| Output order against the shredder's own record of the blocks | 269 of 269 and 512 of 512 transactions, in order |
| Versions decoded | 178 legacy, 331 v0 and 272 v1 |
| Every 3rd data shred dropped before decoding | all 781 transactions recovered from coding shreds |
| Over real UDP: reordered, duplicated, 1 in 4 data shreds dropped | 781 of 781, no duplicates, nothing foreign |
| `static_keys` and `program_ids` against an independent TypeScript decoder | 781 of 781 identical |

## Production notes

- Bound memory. The assembler keeps every slot it has seen; in a long-running process, drop slots that fall a fixed number of slots behind the newest one.
- Read fast, decode elsewhere. If decoding blocks the socket, the kernel buffer fills and packets are dropped. Move parsing to another thread once you go beyond a test.
- Expect duplicates. Your receiver can see the same shred twice, and the same transaction can appear in a block that is later skipped and again in a later block; [deduplicate by signature](https://solanashreds.xyz/docs/pipeline/deduplicate) before you store anything.
- v0 accounts are incomplete here: the line lists static keys only. [Resolving v0 lookup tables](https://solanashreds.xyz/docs/decode/resolve-lookup-tables) adds the rest.
- Recovery rebuilds data but does not re-check the leader's signature over the Merkle root. Add that check if your source of shreds is not one you trust.

## Questions

### Which Rust crate decodes Solana shreds?

`solana-entry` 4.4.1 decodes the block components inside them, and `solana-transaction` the transactions. The shred header itself is simple enough to read by hand, which avoids `solana-ledger` and its RocksDB build.

### Does it decode instruction data?

No. It identifies programs and accounts. Instruction meaning comes next: [decoding Anchor instruction data](https://solanashreds.xyz/docs/decode/anchor-instructions) and the program recipes build on these lines.

### Why are transactions sometimes printed before the slot is complete?

By design. Each block component is decoded as soon as it is whole, so you see the first transactions of a slot early. The [pipeline stages](https://solanashreds.xyz/docs/decode/pipeline) explain the trade-off.

## What this decoder does not do

It reports requested transactions, not outcomes: no success flag, logs, inner instructions or balance changes, because shreds do not carry them. It cannot recover a set that lost more shreds than the set has coding shreds, and it does not resolve lookup tables. Choosing a language on other grounds is covered in [choosing a language](https://solanashreds.xyz/compare/decoder-languages), and a faster parser design in [zero-copy parsing](https://solanashreds.xyz/blog/zero-copy-parsing).

## Next

[Check that shreds reach your server](https://solanashreds.xyz/docs/start/verify-delivery) before you run it, then [run it on a live feed](https://solanashreds.xyz/pricing).
