Decode

A Go decoder using solana-go

A receiver and decoder in one Go program: a goroutine for each stage, bounded channels between them, and solana-go doing the transaction decoding. It keeps up with a fast stream, says plainly what it drops when it cannot, and reports the transaction version the right way.

In Go, decode Solana transactions with solana-go: solana.TransactionFromDecoder reads one legacy, v0 or v1 transaction from a byte stream and stops exactly at its end. Around it, run one goroutine to read UDP, one to assemble shreds into block components and one to decode, joined by bounded channels. Map GetVersion() before printing it.

Stage per goroutine#

The program follows the decoding pipeline one stage per goroutine. The reader only pulls datagrams off the socket. The assembler parses each shred header, keeps data shreds per slot and emits a block component whenever a run of shreds ends with the DATA_COMPLETE flag. The decoder walks the entries in each component and turns every transaction into a JSON line.

Each stage owns its own state, so there are no locks: the assembler's map of slots is touched by one goroutine only. Channels carry finished work forward. If you later move decoding into a separate service, the channel between the assembler and the decoder is where the network goes; decode inside the bot or in a separate service weighs that choice.

Steps#

  1. Read: net.ListenUDP on your port, with a large socket buffer, handing each datagram to a channel.
  2. Parse the header: the variant byte at offset 64 marks a data shred, then slot, index, flags and size.
  3. Assemble: collect data shreds per slot until a run ending in DATA_COMPLETE forms a block component.
  4. Frame the entries: read a u64 entry count; for each entry skip num_hashes and the 32-byte hash, then read how many transactions follow.
  5. Decode: call solana.TransactionFromDecoder once per transaction on the same decoder; it advances past exactly one transaction each time.
  6. Write one JSON line per transaction: slot, signature, version and the programs it calls.

go.mod#

Save it next to main.go, then run go mod tidy to fetch solana-go and record its own dependencies.

Text
module solanashreds-e3/godecoder
go 1.27
require (
github.com/gagliardetto/binary v0.8.0
github.com/gagliardetto/solana-go v1.26.0
)
require (
github.com/blendle/zapdriver v1.3.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/gagliardetto/treeout v0.1.4 // indirect
github.com/goccy/go-json v0.10.6 // indirect
github.com/klauspost/compress v1.18.7 // indirect
github.com/logrusorgru/aurora v2.0.3+incompatible // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/mostynb/zstdpool-freelist v0.0.0-20201229113212-927304c0c3b1 // indirect
github.com/oasisprotocol/curve25519-voi v0.0.0-20251114093237-2ab5a27a1729 // indirect
github.com/streamingfast/logging v0.0.0-20250404134358-92b15d2fbd2e // indirect
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/term v0.46.0 // indirect
)

main.go#

Go
// A Solana transaction decoder for raw shreds, one goroutine per stage, with solana-go v1.26.0.
//
// go run . -port 8001 listen on UDP port 8001
// go run . -file saved.shreds read a saved capture (each datagram prefixed by its u16 little-endian length)
//
// stderr gets "dropped N" every -report interval while N is changing, and once more on exit (Ctrl-C included).
//
// Stages: read datagrams -> assemble block components -> decode transactions -> JSON lines on stdout.
// Data shreds only: a lost data shred costs the block component it belongs to.
package main
import (
"bufio"
"encoding/binary"
"encoding/json"
"flag"
"fmt"
"io"
"log"
"net"
"os"
"os/signal"
"sort"
"sync/atomic"
"syscall"
"time"
bin "github.com/gagliardetto/binary"
"github.com/gagliardetto/solana-go"
)
const dataComplete = 0x40
type component struct {
slot uint64
bytes []byte
}
// Line is one decoded transaction.
type Line struct {
Slot uint64 `json:"slot"`
Signature string `json:"signature"`
Version string `json:"version"`
Programs []string `json:"programs"`
}
// solana-go reports an enum (legacy 0, V0 1, V1 2), not the wire version: map it, never print the number.
var wireVersion = map[solana.MessageVersion]string{
solana.MessageVersionLegacy: "legacy",
solana.MessageVersionV0: "0",
solana.MessageVersionV1: "1",
}
var dropped atomic.Uint64 // datagrams dropped because the assembler fell behind (live mode only)
// Stage 1: read. In live mode a full channel drops the datagram and counts it, so the socket is never blocked.
func readUDP(port int, out chan<- []byte) {
conn, err := net.ListenUDP("udp4", &net.UDPAddr{Port: port})
if err != nil {
log.Fatal(err)
}
_ = conn.SetReadBuffer(64 << 20)
log.Printf("listening on udp/%d", port)
for {
buf := make([]byte, 1280)
n, _, err := conn.ReadFromUDP(buf)
if err != nil {
log.Fatal(err)
}
select {
case out <- buf[:n]:
default:
dropped.Add(1)
}
}
}
func readFile(path string, out chan<- []byte) {
f, err := os.Open(path)
if err != nil {
log.Fatal(err)
}
r := bufio.NewReader(f)
var size [2]byte
for {
if _, err := io.ReadFull(r, size[:]); err != nil {
close(out)
return
}
buf := make([]byte, binary.LittleEndian.Uint16(size[:]))
if _, err := io.ReadFull(r, buf); err != nil {
log.Fatal(err)
}
out <- buf // a file can wait: no drops here
}
}
// Stage 2: assemble data shreds into block components (a run of shreds ending in DATA_COMPLETE).
func assemble(in <-chan []byte, out chan<- component) {
type part struct {
flags byte
payload []byte
}
type slotState struct {
parts map[uint32]part
starts map[uint32]bool
done map[uint32]bool
}
slots := map[uint64]*slotState{}
for buf := range in {
if len(buf) < 88 || (buf[64]&0xf0 != 0x90 && buf[64]&0xf0 != 0xb0) {
continue // coding shreds and anything that is not a shred
}
slot, index := binary.LittleEndian.Uint64(buf[65:]), binary.LittleEndian.Uint32(buf[73:])
flags, size := buf[85], int(binary.LittleEndian.Uint16(buf[86:]))
if size < 88 || size > len(buf) {
continue
}
s := slots[slot]
if s == nil {
s = &slotState{map[uint32]part{}, map[uint32]bool{0: true}, map[uint32]bool{}}
slots[slot] = s
}
if _, dup := s.parts[index]; dup {
continue
}
s.parts[index] = part{flags, buf[88:size]}
if flags&dataComplete != 0 {
s.starts[index+1] = true
}
starts := make([]uint32, 0, len(s.starts))
for st := range s.starts {
starts = append(starts, st)
}
sort.Slice(starts, func(i, j int) bool { return starts[i] < starts[j] })
for _, start := range starts {
if s.done[start] {
continue
}
end := start
for p, ok := s.parts[end]; ok && p.flags&dataComplete == 0; p, ok = s.parts[end] {
end++
}
if _, ok := s.parts[end]; !ok {
continue
}
s.done[start] = true
var b []byte
for i := start; i <= end; i++ {
b = append(b, s.parts[i].payload...)
}
out <- component{slot, b}
}
}
close(out)
}
// Stage 3: decode. solana-go reads each transaction off the stream and stops exactly at its end,
// for legacy, v0 and v1, so the entries can be walked without measuring lengths by hand.
func decode(in <-chan component, out *bufio.Writer) {
enc := json.NewEncoder(out)
for c := range in {
d := bin.NewBinDecoder(c.bytes)
entries, err := d.ReadUint64(bin.LE)
if err != nil {
continue
}
for e := uint64(0); e < entries; e++ { // a zero count is a header or footer marker
if err := d.SkipBytes(8 + 32); err != nil { // num_hashes and the entry hash
break
}
count, err := d.ReadUint64(bin.LE)
if err != nil {
break
}
for t := uint64(0); t < count; t++ {
tx, err := solana.TransactionFromDecoder(d)
if err != nil {
fmt.Fprintf(os.Stderr, "slot %d: undecodable transaction: %v\n", c.slot, err)
break
}
seen, programs := map[string]bool{}, []string{}
for _, ix := range tx.Message.Instructions {
p := tx.Message.AccountKeys[ix.ProgramIDIndex].String()
if !seen[p] {
seen[p] = true
programs = append(programs, p)
}
}
enc.Encode(Line{c.slot, tx.Signatures[0].String(), wireVersion[tx.Message.GetVersion()], programs})
}
}
out.Flush()
}
}
// reportDrops writes the drop count to stderr whenever it has changed since the last tick.
func reportDrops(every time.Duration) {
last := uint64(0)
for range time.Tick(every) {
if n := dropped.Load(); n != last {
log.Printf("dropped %d", n)
last = n
}
}
}
func main() {
port := flag.Int("port", 8001, "UDP port to listen on")
file := flag.String("file", "", "read a saved capture instead of the socket")
queue := flag.Int("queue", 8192, "datagrams the reader may hold for the assembler before it drops")
report := flag.Duration("report", 10*time.Second, "how often to print the drop count while it changes")
flag.Parse()
datagrams := make(chan []byte, *queue) // bounded: see the drop policy in readUDP
components := make(chan component, 256)
if *file != "" {
go readFile(*file, datagrams)
} else {
go readUDP(*port, datagrams)
go reportDrops(*report)
stop := make(chan os.Signal, 1)
signal.Notify(stop, os.Interrupt, syscall.SIGTERM)
go func() {
<-stop
log.Printf("dropped %d", dropped.Load())
os.Exit(0)
}()
}
go assemble(datagrams, components)
decode(components, bufio.NewWriter(os.Stdout))
log.Printf("dropped %d", dropped.Load())
}

Bounded channels and what to drop when full#

Every channel has a fixed size, so a slow stage cannot make memory grow without limit. The question is what happens when one fills up. Between the assembler and the decoder the program simply waits: the assembler blocks for a moment, and the queue in front of it absorbs the delay.

Between the socket and the assembler it does not wait. If the reader blocked, the kernel's socket buffer would fill and the kernel would start dropping packets with no record of it. Instead the reader drops the datagram itself and counts it in dropped, so the loss is visible in your own numbers. The program writes a timestamped line ending in dropped N to stderr every 10 seconds while the count is changing (set the interval with -report) and once more when it exits, Ctrl-C included. Feed that line, or the counter itself, into your metrics; a rising value means the assembler or decoder needs more CPU. The -queue flag sets how many datagrams may wait for the assembler. In file mode nothing is ever dropped, because a file can wait.

The version check#

solana-go's Message.GetVersion() returns an enum, not the number on the wire: legacy is 0, v0 is 1 and v1 is 2. Printing it directly would label every v0 transaction as version 1. The wireVersion map turns the enum into the names Solana uses, and you should compare against solana.MessageVersionV1 and its siblings rather than against raw numbers.

Fixture run#

Inputs: the two shared fixture blocks used across these docs, real mainnet slots cut into shreds with the Agave 4.4.1 shredder. Test: node test.mjs (13 checks) with Go 1.27.4 and solana-go v1.26.0, run three times in a row.

CheckResult
go vet and gofmtclean
-file with each saved slot269 of 269 and 512 of 512 transactions, in block order
Version and program list against @solana/kit, a separate decoderidentical on all 781 transactions
Local replay over UDP at 20,000 datagrams a second (a test rate, not the feed's rate), shuffled, every 5th sent twice269 of 269 and 512 of 512, each printed once
Drop counter: -file runs and the normal replay, then -queue 1 at 200,000 datagrams a second (a test rate, not the feed's rate)dropped 0, dropped 0, then a rising count in the periodic report and the same final count on Ctrl-C
TransactionFromDecoder position after one transaction followed by more bytesexactly at its end for all 178 legacy, 331 v0 and 272 v1 transactions

The last row matters for framing. A decoder that reads past the end of a transaction cannot walk an entry batch, so this was checked on its own before relying on it.

What this decoder leaves out#

It uses data shreds only, so a lost data shred costs every transaction in its block component; the Rust decoder rebuilds lost shreds from coding shreds. It keeps every slot it has seen, so a long-running process should drop slots that fall far behind the newest one. Program and account keys are the static keys; a v0 transaction's lookup-table accounts need lookup table resolution. Execution results are never in shreds.

Questions#

Which Go function decodes a Solana transaction?#

solana.TransactionFromDecoder from solana-go reads one transaction from a byte decoder; solana.TransactionFromBytes does the same for a byte slice you already cut out. Version 1.26.0 handles legacy, v0 and v1.

Why does GetVersion() say 1 for a v0 transaction?#

Because it returns an enum, not the wire version: legacy is 0, v0 is 1 and v1 is 2. Map it to names before you print or store it.

How do I know if the decoder is falling behind?#

Read stderr: the program prints a timestamped line ending in dropped N every 10 seconds while the count changes, and the final count on exit. It counts datagrams the reader discarded because the next stage was full; anything above zero on a steady stream means you need more processing capacity.

Next#

Which language should decode your transactions compares Go with the others, versioned transactions explains the three formats behind the version check, and the Python decoder covers offline captures. To run this on your own stream, see pricing.

Sources#

Raw shreds on your own server

Ultra Low Latency Shreds is $200 and Standard Shreds is $100. You pay in crypto, tell us which IPv4 address and UDP port should receive the stream, and we wire it up by hand.