Canonical: https://solanashreds.xyz/blog/idl-drift

# IDL drift: when a program upgrade breaks your decoder

Programs you decode will change. A decoder that applies yesterday's layout to today's bytes rarely crashes; it writes wrong rows. Here is a policy that turns drift into an alert.

When an Anchor IDL changed and your decoder broke without an error, it kept applying the old layout to new bytes. Pin each IDL to the slot range it covers, watch for upgrades of every program you decode, and treat an unknown discriminator or an unexpected data length as an alert, never as a row to guess at.

## Why decoders break without an error

The worst version of this story is the one where nothing failed. A program adds a field to an argument struct, and your decoder, still using the old struct, reads the first part of the new data and stops. If it ignores trailing bytes, it succeeds. The row it writes has the right shape and the wrong meaning.

Three kinds of change cause most drift:

- **Added arguments.** New fields appended to an instruction's arguments. Old clients keep sending the short form, so both shapes appear in the stream at once.
- **Changed types or order.** A field widened from u32 to u64, or two fields swapped. Every later offset moves.
- **New or replaced instructions.** A new instruction with a new discriminator, or an old one superseded by a second version under a new name.

Since Anchor 0.31, programs can also set custom discriminators, so a decoder that computes them from instruction names can stop matching after an upgrade even when the arguments did not change.

## Where IDLs live and how they change

An IDL can come from three places, and they do not always agree:

- **The program's repository**, published by its team. Convenient, but a repository copy can lag behind the deployed program.
- **An on-chain IDL account**, which Anchor programs can publish so the IDL sits next to the program itself. For an Anchor program this is usually the most current copy.
- **An SDK package**, where some teams ship the IDL inside their client library.

When two copies disagree, the deployed program is what matters. Check a sample of recent instructions against each copy and keep the one that decodes them all cleanly. Record where every IDL you use came from and the commit, version or slot at which you took it.

## Pinning layouts to slot ranges

Do not hold one IDL per program. Hold a list of layouts per program, each valid for a range of slots.

| Field | Example value | Purpose |
|---|---|---|
| program id | the program's address | Which program this layout applies to |
| valid from slot | the slot of the upgrade that introduced it | Lower bound, inclusive |
| valid to slot | empty while current | Upper bound, set when the next upgrade lands |
| source | repository commit, package version or on-chain account | Where the IDL came from |
| idl hash | a hash of the IDL file | Detects a silent change in the file itself |
| added by | a person or job | Who approved it |

The decoder selects the layout by the transaction's slot. Old transactions in a backfill then decode with the layout of their time, and a reprocessing job gives the same answer it gave the first time.

## Detecting a change before it costs you

- **Watch for upgrades.** An upgrade of a program you decode arrives as an `Upgrade` instruction to the upgradeable loader. [Watching program upgrades](https://solanashreds.xyz/blog/watching-program-upgrades) shows how to spot it in the stream.
- **Watch the IDL sources.** Check the repositories and on-chain IDL accounts you depend on for changes, and compare hashes with the ones in your layout table.
- **Watch the data.** Track, per program, the share of instructions whose discriminator you do not know and the share whose data length does not match any known layout. Both should be zero almost all the time.
- **Keep fixtures from each layout.** When a new layout lands, add fixtures from after the upgrade and keep the old ones. [Golden fixtures](https://solanashreds.xyz/blog/regression-tests-for-decoders) explains how.

## Failing loudly on unknown discriminators

When the decoder meets an instruction it cannot place, it has three choices: guess, skip or flag. Guessing is how wrong rows get written. Skipping hides the problem. Flagging is the only safe option.

Write the raw instruction to an "undecoded" table with the program id, the slot and the first 8 bytes of its data, and increment a counter. Alert when the counter rises for a program that normally has none. Once you add the new layout, reprocess the undecoded rows; because you kept the raw bytes, nothing is lost.

Treat data length the same way. If an instruction's data ends exactly at an argument boundary, record the missing arguments as absent. If it is longer than any known layout, flag it.

## What versioning cannot do for you

A layout table only knows the layouts you put in it. The first transactions after an upgrade will reach your decoder before you have read the new IDL, and they will be flagged rather than decoded. That is the intended behaviour, but it means a gap until someone adds the layout. Versioning also says nothing about what happened when the transaction ran: a correctly decoded instruction is still only a request until its outcome is confirmed.

## Related reading

[Reading an IDL](https://solanashreds.xyz/learn/reading-an-idl) shows which parts of an IDL a decoder actually uses, and [decoding Anchor instructions](https://solanashreds.xyz/docs/decode/anchor-instructions) builds the IDL driven decoder that this policy wraps. If decoded events feed other systems, [event schemas that last](https://solanashreds.xyz/blog/event-schemas-that-last) shows how to version the interpretation as well as the layout. Drift is easiest to catch on live data, so a [Standard Shreds](https://solanashreds.xyz/standard-shreds) destination is a practical place to run these checks.

When a layout changes underneath you, the first symptom is often one of [the Borsh errors catalogued here](https://solanashreds.xyz/blog/borsh-pitfalls).
