Canonical: https://solanashreds.xyz/blog/zero-copy-parsing

# Zero copy parsing for Solana transactions

A transaction arrives as bytes you already hold. A borrowed view reads signatures, keys and instructions in place, and hands out slices instead of fresh vectors. Here is a tested parser and the cases where you still need to copy.

Zero copy Solana transaction parsing means reading signatures, keys and instructions as borrowed slices of the buffer the transaction arrived in, instead of building new vectors for each field. In Rust that is a view struct tied to the buffer's lifetime. It saves allocation on the hot path, and you copy only what must outlive the packet.

## What zero copy means for a transaction

A decoder that deserialises each transaction into an owned structure allocates a vector for the signatures, another for the keys, one per instruction for its accounts and data, and more for lookup tables. Most of those bytes are read once, checked against a filter and thrown away.

A borrowed view keeps a reference to the original bytes and records where each field sits. Signatures and keys become slices of fixed size arrays, instruction data becomes a slice, and nothing is copied until a caller decides a transaction is worth keeping. The cost of this design is lifetimes: the view cannot outlive the buffer, so the buffer has to stay alive while you work with it.

Legacy and v0 transactions share one layout: a compact length and the signatures, then the message. A v0 message starts with a prefix byte of `0x80`, and adds a list of lookup table references at the end. Version 1 transactions, defined in SIMD-0385, start with a version byte of 129 and put the signatures last, so they need a separate parser. The one below detects and rejects them rather than misreading them.

## Building the borrowed parser, step by step

1. Read the compact length of the signature list, then take that many 64 byte signatures as one slice.
2. Look at the next byte: if its top bit is set, it is a version prefix, and only version 0 is accepted here.
3. Take the three byte header, the compact length and the 32 byte static keys, and the recent blockhash.
4. Walk the instructions once to check every length fits inside the buffer, and remember where they start and end.
5. For v0, do the same for the lookup table references.
6. Return the view together with the number of bytes used, because transactions sit back to back inside an entry.
7. Hand out instructions and lookups through iterators that read from the remembered slices.

## The parser

The code uses only the standard library and was compiled and tested with Rust 1.93. `as_chunks` turns a byte slice into a slice of fixed size arrays without copying; on toolchains older than 1.88, `bytemuck::cast_slice` does the same job.

```rust file="txview.rs"
#[derive(Debug, PartialEq)]
pub enum Version { Legacy, V0 }

#[derive(Debug, PartialEq)]
pub enum ParseError { Truncated, BadLength, Unsupported(u8) }

fn compact_u16(buf: &[u8], pos: &mut usize) -> Result<usize, ParseError> {
    let mut value = 0usize;
    for i in 0..3 {
        let b = *buf.get(*pos).ok_or(ParseError::Truncated)?;
        *pos += 1;
        if i == 2 && b > 0x03 { return Err(ParseError::BadLength); }
        if i > 0 && b == 0 { return Err(ParseError::BadLength); }
        value |= ((b & 0x7f) as usize) << (7 * i);
        if b & 0x80 == 0 { return Ok(value); }
    }
    Err(ParseError::BadLength)
}

fn take<'a>(buf: &'a [u8], pos: &mut usize, n: usize) -> Result<&'a [u8], ParseError> {
    let end = pos.checked_add(n).ok_or(ParseError::BadLength)?;
    let s = buf.get(*pos..end).ok_or(ParseError::Truncated)?;
    *pos = end;
    Ok(s)
}

fn chunks<'a, const N: usize>(buf: &'a [u8], pos: &mut usize, count: usize) -> Result<&'a [[u8; N]], ParseError> {
    let bytes = take(buf, pos, count.checked_mul(N).ok_or(ParseError::BadLength)?)?;
    let (whole, rest) = bytes.as_chunks::<N>();
    debug_assert!(rest.is_empty());
    Ok(whole)
}

pub struct Instruction<'a> { pub program_id_index: u8, pub accounts: &'a [u8], pub data: &'a [u8] }
pub struct Lookup<'a> { pub table: &'a [u8; 32], pub writable: &'a [u8], pub readonly: &'a [u8] }

/// Borrowed view of a legacy or v0 transaction. Nothing is copied out of `buf`.
pub struct TxView<'a> {
    pub signatures: &'a [[u8; 64]],
    pub version: Version,
    pub header: &'a [u8; 3],
    pub static_keys: &'a [[u8; 32]],
    pub recent_blockhash: &'a [u8; 32],
    ix_bytes: &'a [u8],
    ix_count: usize,
    lookup_bytes: &'a [u8],
    lookup_count: usize,
}

impl<'a> TxView<'a> {
    /// Parses one transaction from the front of `buf` and returns it with the number of bytes it used,
    /// so a caller can walk the transactions packed one after another inside an entry.
    pub fn parse(buf: &'a [u8]) -> Result<(Self, usize), ParseError> {
        if buf.first() == Some(&0x81) { return Err(ParseError::Unsupported(0x81)); } // v1: different layout
        let mut pos = 0;
        let nsig = compact_u16(buf, &mut pos)?;
        let signatures = chunks::<64>(buf, &mut pos, nsig)?;
        let version = if *buf.get(pos).ok_or(ParseError::Truncated)? & 0x80 != 0 {
            let v = buf[pos] & 0x7f;
            if v != 0 { return Err(ParseError::Unsupported(buf[pos])); }
            pos += 1;
            Version::V0
        } else { Version::Legacy };
        let header: &[u8; 3] = take(buf, &mut pos, 3)?.try_into().unwrap();
        let nkeys = compact_u16(buf, &mut pos)?;
        let static_keys = chunks::<32>(buf, &mut pos, nkeys)?;
        let recent_blockhash: &[u8; 32] = take(buf, &mut pos, 32)?.try_into().unwrap();
        let ix_count = compact_u16(buf, &mut pos)?;
        let ix_start = pos;
        for _ in 0..ix_count {
            take(buf, &mut pos, 1)?;
            let n = compact_u16(buf, &mut pos)?; take(buf, &mut pos, n)?;
            let n = compact_u16(buf, &mut pos)?; take(buf, &mut pos, n)?;
        }
        let ix_bytes = &buf[ix_start..pos];
        let (mut lookup_bytes, mut lookup_count) = (&buf[pos..pos], 0);
        if version == Version::V0 {
            lookup_count = compact_u16(buf, &mut pos)?;
            let start = pos;
            for _ in 0..lookup_count {
                take(buf, &mut pos, 32)?;
                let n = compact_u16(buf, &mut pos)?; take(buf, &mut pos, n)?;
                let n = compact_u16(buf, &mut pos)?; take(buf, &mut pos, n)?;
            }
            lookup_bytes = &buf[start..pos];
        }
        Ok((TxView { signatures, version, header, static_keys, recent_blockhash, ix_bytes, ix_count, lookup_bytes, lookup_count }, pos))
    }

    pub fn instructions(&self) -> impl Iterator<Item = Instruction<'a>> + 'a {
        let (buf, count) = (self.ix_bytes, self.ix_count);
        let mut pos = 0;
        (0..count).map(move |_| {
            // the bounds were checked once in parse(), so these cannot fail here
            let program_id_index = buf[pos]; pos += 1;
            let n = compact_u16(buf, &mut pos).unwrap(); let accounts = take(buf, &mut pos, n).unwrap();
            let n = compact_u16(buf, &mut pos).unwrap(); let data = take(buf, &mut pos, n).unwrap();
            Instruction { program_id_index, accounts, data }
        })
    }

    pub fn lookups(&self) -> impl Iterator<Item = Lookup<'a>> + 'a {
        let (buf, count) = (self.lookup_bytes, self.lookup_count);
        let mut pos = 0;
        (0..count).map(move |_| {
            let table: &[u8; 32] = take(buf, &mut pos, 32).unwrap().try_into().unwrap();
            let n = compact_u16(buf, &mut pos).unwrap(); let writable = take(buf, &mut pos, n).unwrap();
            let n = compact_u16(buf, &mut pos).unwrap(); let readonly = take(buf, &mut pos, n).unwrap();
            Lookup { table, writable, readonly }
        })
    }
}
```

The tests build a legacy and a v0 transaction by hand, check every field, and assert that the instruction data slice points into the original buffer rather than a copy:

```rust file="txview.rs (tests)"
#[cfg(test)]
mod tests {
    use super::*;
    fn sample(v0: bool) -> Vec<u8> {
        let mut b = vec![1u8];                 // one signature
        b.extend([7u8; 64]);
        if v0 { b.push(0x80); }                 // version prefix
        b.extend([1, 0, 1]);                    // header
        b.push(2); b.extend([1u8; 32]); b.extend([2u8; 32]); // two static keys
        b.extend([9u8; 32]);                    // recent blockhash
        b.push(1);                              // one instruction
        b.push(1); b.push(1); b.push(0);        // program index 1, accounts [0]
        b.push(4); b.extend([2, 0, 0, 0]);      // data
        if v0 { b.push(1); b.extend([5u8; 32]); b.push(1); b.push(3); b.push(0); }
        b
    }
    #[test] fn parses_legacy_and_v0_without_copying() {
        for v0 in [false, true] {
            let bytes = sample(v0);
            let (tx, used) = TxView::parse(&bytes).unwrap();
            assert_eq!(used, bytes.len());
            assert_eq!(tx.version, if v0 { Version::V0 } else { Version::Legacy });
            assert_eq!(tx.signatures.len(), 1);
            assert_eq!(tx.static_keys[1], [2u8; 32]);
            let ix: Vec<_> = tx.instructions().collect();
            assert_eq!((ix[0].program_id_index, ix[0].accounts, ix[0].data), (1, &[0u8][..], &[2u8, 0, 0, 0][..]));
            assert!(std::ptr::eq(ix[0].data.as_ptr(), bytes[bytes.len() - if v0 { 40 } else { 4 }..].as_ptr()));
            assert_eq!(tx.lookups().count(), if v0 { 1 } else { 0 });
        }
    }
    #[test] fn rejects_truncated_and_v1() {
        let bytes = sample(true);
        assert_eq!(TxView::parse(&bytes[..bytes.len() - 1]).err(), Some(ParseError::Truncated));
        assert_eq!(TxView::parse(&[0x81, 0, 0]).err(), Some(ParseError::Unsupported(0x81)));
    }
    #[test] fn compact_u16_multi_byte() {
        let mut p = 0; assert_eq!(compact_u16(&[0x80, 0x01], &mut p), Ok(128));
        let mut p = 0; assert_eq!(compact_u16(&[0xff, 0xff, 0x03], &mut p), Ok(0xffff));
        let mut p = 0; assert_eq!(compact_u16(&[0xff, 0xff, 0x04], &mut p), Err(ParseError::BadLength));
    }
}
```

Run them with `rustc --edition 2021 --test txview.rs -o t && ./t`.

## Compact u16 lengths without copying

Every list in a transaction is prefixed by a compact u16: one to three bytes, seven bits of value per byte, with the top bit meaning "another byte follows". The third byte may only use its two lowest bits, otherwise the value would not fit in sixteen bits. The `compact_u16` function above enforces that rule, rejects alias encodings such as a trailing zero byte, and returns an error instead of reading past the buffer.

Reading the length is the only arithmetic the parser does. Everything else is slicing at offsets, and each `take` checks bounds with `get`, so a malformed length produces an error rather than a panic. That matters because the decoder will meet [malformed input](https://solanashreds.xyz/blog/your-decoder-will-meet-garbage) sooner or later.

## When you have to copy after all

- **Storage and queues.** Anything you send to Kafka, a database or another thread that outlives the receive buffer needs owned bytes. Copy at that boundary, once, and only for the transactions you keep.
- **Lookup tables.** Resolving a v0 transaction's addresses means joining static keys with addresses from table accounts that live elsewhere. The resolved account list is new data, so it is owned, but you can build it only after the filter has passed.
- **Buffer reuse.** If your receive loop reuses one buffer per datagram, a view must be finished before the next read. Decide per stage whether to process in place or to move the bytes into an owned frame.

## Testing the borrowed parser against a reference

A hand written parser needs a reference to agree with. Decode the same transactions with the official `solana-transaction` crate, version 6.0.0, and compare signatures, keys, instructions and lookups field by field. Feed it real legacy and v0 transactions from finalized blocks, v1 transactions to check that they are rejected, plus deliberately truncated copies. The [Rust decoder guide](https://solanashreds.xyz/docs/decode/rust) builds that reference path, and [transaction structure](https://solanashreds.xyz/learn/solana-transaction-structure) explains each field the view exposes.

## Where zero copy is not worth it

If your decoder keeps most transactions, or the time goes to lookup resolution and writes rather than parsing, borrowing saves little and adds lifetime complexity. Measure first: [what decoding costs in CPU](https://solanashreds.xyz/blog/what-decoding-costs-in-cpu) shows how to find the stage that dominates. The [parsing libraries comparison](https://solanashreds.xyz/compare/shred-parsing-libraries) covers maintained crates that may already be fast enough for you.

## Next steps

Start with the [quickstart](https://solanashreds.xyz/docs/quickstart) to get packets into a decoder, then swap in the borrowed parser where profiling says it pays.

Zero copy parsing gets harder once accounts live in lookup tables, which is [the lookup table problem](https://solanashreds.xyz/blog/the-lookup-table-problem) in a nutshell.

Parsing less is often better than parsing faster, which is why [where you filter](https://solanashreds.xyz/blog/filter-early) matters as much as how you parse.
