The Budget

A transformer lets every token attend to every other token. That is n² pairwise relationships for n tokens, and the model's capacity to hold them is fixed. So attention behaves like a budget: every token you add spends some of it, and the tokens already there get a little less.

The failure this produces is called context rot — as the token count rises, the model's ability to accurately recall information from the window falls. It is not a cliff. It is a gradient, which is worse, because the system keeps working while quietly getting less precise. You do not get an error. You get an answer that is slightly wrong more often.

The consequence for anyone writing context is a single instruction:

Find the smallest possible set of high-signal tokens that maximise the likelihood of the outcome you want.

Note what that does not say. It does not say "be brief". Minimal is not short — a complex constraint still needs the detail that makes it enforceable. It says every token should be earning its place.

Two prompts of the same length can differ by a factor of three in how much they actually say. The usual culprit is restatement — the same instruction delivered three times in three registers, because emphasis feels like insurance.

It is not insurance. It costs budget and it competes with itself: when four things are marked critical, none of them is.

So the thing to measure is not length but density — distinct content words per token — and redundancy, the share of your phrasing that repeats. Cutting a prompt should raise density. If you cut and density falls, you removed content and kept the padding.