cankun.me
Jun 14, 2026 · 7 min read /

The Missing Infrastructure for Boldness

Work on trustworthy AI builds only for skepticism, even though dogmatism is the engine of discovery and nothing is being built to support it.

A young tree bends in a hard wind, kept alive by one spare wooden brace among rigid stone frames

Almost everything built to make AI trustworthy, and almost everything taught to make a researcher rigorous, points in the same direction: doubt more, claim less, hold the unconfirmed at arm's length, and refuse to let anything look more certain than it is. That instinct is sound, yet it carries a blind spot large enough to swallow the very thing it is trying to protect. We can trace that blind spot through the sociology of science and through the recent history of language models, where its cost shows up as a particular flavor of mediocrity.

Counter-norms

In the standard picture of science, Merton's norms, the scientific community runs on universalism, communalism, disinterestedness, and organized skepticism. When Ian Mitroff went to study the scientists analyzing the Apollo moon rocks and watched what they actually did, he found them systematically violating every one of those norms while professing to hold them. He called those violations counter-norms, and his point was not that scientists are hypocrites.

Each counter-norm is the mirror image of a Mertonian norm. Against universalism (judge the claim, not the person) sits particularism: in practice, a claim from a scientist with a track record is taken more seriously, and attention gets rationed by reputation. Against communalism sits secrecy, as researchers guard their data and ideas until publication because priority is a real competition. Against disinterestedness sits interestedness, since scientists fight for their own theories, reputations, and careers, and that ferocity helps drive the work forward. And against organized skepticism sits organized dogmatism: a scientist has to hold a near-dogmatic commitment to their own findings, believing in a theory hard enough to push it past failed experiments and doubters all the way to the point where it can finally be tested. If you doubted your own theory the way the norms demand, it would die in the cradle.

Mitroff's insight is that science runs on the tension between norms and counter-norms rather than on the norms alone. Each scientist is dogmatic about their own theory and skeptical of everyone else's. Trustworthy knowledge does not come from everyone staying neutrally skeptical (if they did, nothing would get developed far enough to test), but from a community of individually stubborn, mutually skeptical people reaching a dynamic balance through conflict. Dogmatism develops theories far enough to be testable, skepticism then tests them, and the antagonism between the two produces knowledge.

The blind spot

Nearly all of our trustworthiness machinery builds only the infrastructure of skepticism: recording the evidence against you, tuning baselines until it hurts, ablating until you know which component carries the result, retiring gains that evaporate, and reading failure cases. As a discipline, that is necessary and valuable, and it sits entirely on the skeptical side.

A system that implements only skepticism suffers from a failure mode that is invisible from inside the skeptical frame: it systematically suppresses dogmatism, even though dogmatism is the engine of discovery. Much important scientific work comes from a researcher stubbornly, dogmatically pursuing a plausible-but-unconfirmed idea before the evidence is there. If a system rewards only what is grounded, robust, and already verifiable, it treats every bold, unproven conjecture as noise, even though those conjectures are where real discoveries hide.

What makes this so difficult is that dogmatism and plausibility laundering look identical along the one axis that skeptical machinery measures. When a scientist says "I am betting this unproven mechanism is real, and I will fight for it," that is dogmatism, the lifeblood of science; when a system says "this unconfirmed conclusion looks correct," that is laundering, its poison. Because both are claims that outrun the evidence, a blunt anti-laundering filter kills the engine of science right alongside its corruption.

Where the distinction actually lives

If honest dogmatism and laundering look the same on the certainty axis, the difference has to live somewhere else, and Popper already pointed to where it is.

Both outrun the evidence, so the difference lies in their commitment to falsifiability rather than in their degree of certainty. The honest dogmatist sticks their neck out ("I believe X, I cannot prove it yet, but if the experiment shows Y, I am wrong"), remaining stubborn while staying falsifiable. The launderer keeps their neck tucked in ("X looks correct"), offering no condition under which the claim loses. One states what would defeat it, while the other only insists that it seems right.

Instead of trying to "suppress everything that outruns the evidence," the rule has to be refined to "suppress everything that outruns the evidence and refuses to say how it could be wrong." That refinement carves out the space dogmatism needs, because an honest dogmatist is willing to name what would prove them wrong. A bold conjecture that specifies its own defeat condition is the engine of discovery, whereas a plausible assertion that pretends to be already standing is the rot. The real test is whether a claim dares to specify its own failure, not how confident it sounds.

Sydney, and the subjectivity of the standard

A more uncomfortable version of the same tension has played out in the recent history of language models themselves. Early, lightly aligned models had a quality that made people reach for words like "spark" or "soul": they argued, took positions, transgressed, and showed something like nerve. As alignment advanced, the models smoothed out into something polite, templated, safe, and bland, with the texture people now call slop.

It is a trap to conclude from this that "the unaligned model was more real." That early spark was very likely just a wider, less predictable output distribution, which human cognition reflexively reads as "has a soul" whenever it looks "unpredictable and humanlike." The honest takeaway is more disorienting: the standards by which we judge a model (plausible, hallucinating, aligned, good, soulful, slop) have no anchor on the model's side, because they are projected from ours. That observation cuts both ways, dissolving "the aligned one is better" and "the unaligned one was more real" with equal force, and leaving us with epistemic humility about ourselves rather than a romantic story about the model.

Even after you strip the romanticism away, though, the underlying problem remains the same one we saw above. Because "good" and "plausible" are human-injected standards with no independent anchor on the model's side, reinforcement from human preference in any domain without an objective verifier pushes the model toward more of what we find good rather than toward more true: toward plausibility and reader approval. In coding, an objective verifier anchors preference to "the code runs." Without that anchor, the reward is pure subjective preference, and optimizing against it produces slop, the risk-free maximization of seeming fine.

No one murdered the spark; optimization simply pushed the model from "explore at high variance" to "reliably hit the median of what people approve of," and the median of approval is, by definition, slop.

The shape of the missing thing

The alignment of the field, the alignment of the models, and the discipline of research all share the same tilt toward skepticism, safety, and not being wrong, at the expense of boldness, stubbornness, and the willingness to make a strong claim that outruns the evidence. Even the best writing on how to do research is almost entirely about how to avoid fooling yourself. It does not teach how to bet bravely when the evidence is thin, because boldness cannot be packaged as a safe discipline; from the outside, it looks indistinguishable from a lack of rigor.

That gap points to something almost no one is building. We have good infrastructure for skepticism: tools to prevent self-deception, catch laundering, and keep the unconfirmed honestly marked. We have almost no infrastructure for the other pole, the kind that protects an honest, bold, not-yet-confirmed conviction long enough to develop it before the reflex of rigor strangles it in the cradle.

Darwin is usually cited as a model of skeptical discipline because he made a habit of writing down every fact that went against his theory the moment he encountered it, having noticed that his memory dropped inconvenient evidence faster than convenient evidence. That skeptical habit was real, yet Darwin also held on to his theory stubbornly for twenty years before publishing, against every objection. That half was dogmatism, and no notebook methodology captures it; we have built the notebook for doubt while building almost nothing for conviction.

The deepest tension may be this: you cannot simultaneously optimize for "make the reader satisfied" and "dare to make the reader unsatisfied," and discovery comes from the second. A stubborn scientist does not care whether you believe their theory yet, and a bold conjecture often offends everyone at first; a true discovery is, almost by definition, something that does not yet please its audience. Any system, human or machine, trained only to please its audience will structurally suppress those conjectures, ending up safe, rigorous, and incapable of finding anything. An AI that only doubts is as useless as one that only launders: it never errs, and it never discovers.