cankun.me

What Survives Scale

Jun 14, 2026

There is a question underneath most decisions about what to build in AI right now, and it is rarely asked in plain form: as model capability grows, what becomes worthless, and what does not?

The cleanest articulation of the threat comes from Rich Sutton's bitter lesson. The history of AI keeps teaching the same thing: general methods that ride on computation, search and learning, eventually overtake methods that encode human understanding of a problem. Researchers cannot resist pouring their domain insight into a system. It works in the short term. It becomes the ceiling in the long term, and then a method that simply scales compute walks past it.

If that is true, then the only rational thing to do is to build the part that scale does not eat. Which requires knowing how to tell the difference. And the obvious way to tell the difference is wrong in an instructive way.

The first cut, and why it fails

The tempting discriminator is: encoding a method gets eaten; encoding the structure of the world does not. A workflow that says "first do literature review, then generate hypotheses, then critique" encodes a method, a human prior about how the work should proceed, and a stronger model will dissolve it by decomposing the task itself. Whereas the fact that conclusions rest on evidence, that evidence has a source, that state changes over time, that is world-structure, and it stays.

This is a good heuristic. It is not a law, and it fails at exactly the place you need it most.

It fails because world-structure itself gets eaten. The history of science is a graveyard of confidently-held ontologies. Phlogiston was world-structure. The ether was world-structure. Vital force was the way the world was assumed to be carved at the joints. Each was an account of what kinds of things exist; and each was overturned. So "ontology survives, method dies" claims too much. Ontologies die all the time.

The better cut

What actually survives is narrower and stranger than "world-structure." It is the class of epistemic constraints; and the reason they survive is not that they describe the world correctly, but that a stronger model does not make them unnecessary.

Take the rule that you must not let something unverified look verified. A model a hundred times more capable does not retire this rule. It still cannot collapse the gap between a map and the territory; it lives on the map side of that gap like everything else. Stronger models make better maps. They do not make the distinction between "agreed" and "true" obsolete, because that distinction is not about how good the map is: it is about the fact that the map is a map.

So the discriminator is not method versus world-structure. It is domain shortcut versus epistemic constraint. A domain shortcut encodes how some problem is currently solved; it gets absorbed. An epistemic constraint encodes the irreducible structure of not-yet-knowing (provenance, the difference between consensus and truth, honest accounting of what rests on what), and it does not get absorbed, because no amount of capability removes the condition it responds to.

This is sharper because it explains the failure of the first cut. Ontologies get eaten when they are really domain shortcuts wearing the costume of world-structure: phlogiston was a particular era's best guess about combustion, dressed up as the nature of fire. The constraints that survive are the ones that encode the gap between any model and the world, not any particular model of it.

The unstable corner

There is one place where even this cleaner cut rests on a bet, and honesty requires naming it.

The reason epistemic constraints survive is that the map-territory gap is permanent. And the reason the gap is permanent is that some verifiers are irreducible: there exist questions whose ground truth requires the real world, in real time, at irreducible complexity, to answer. If that were false, if the territory could be fully simulated, if a good enough model could always stand in for reality's reply, then the gap would slowly close, "agreed" would creep toward "true," and the surviving constraints would become as quaint as some discarded ontology.

So the whole edifice rests on a single empirical wager: that the territory contains a residue no map can fully replace. There is reason to believe it. Any simulation is itself a map, so using simulation to stand in for the territory is using a map to verify a map: circular. But this is a structural argument from a premise about the world, not a theorem, and it could be overturned in some specific domain by a single empirical fact.

The honest position is therefore: the discriminator works, and the thing it protects is durable, as long as the territory keeps a residue. Bet that it does. But hold the bet as a bet.

Why this matters for what you build

The practical consequence is that the most defensible thing to build is rarely the most impressive-looking thing. The impressive thing, the system that autonomously does the whole job, is almost always a stack of domain shortcuts that look necessary right now precisely because they fill a gap in general capability. The gap is closing. The shortcuts have an expiration date written on them in invisible ink.

The durable thing is the scaffolding that a more capable model would still need: the honest record of where things stand, the refusal to let the unconfirmed pass as confirmed, the structure that lets work accumulate without quietly corrupting itself. It is less exciting. It is also the part that grows more valuable as the model gets stronger, rather than less, because a stronger model produces more work that needs to be kept honest.

The deepest version of the bitter lesson may be this: scale does not just beat hand-coded knowledge. It reveals, one capability jump at a time, which of the things we called structure were only scaffolding we built because we did not have enough compute to do without it.

← Back to Writing