The Method Is the Artifact

How I govern a research program that might be wrong

Most research programs and most engineering programs fail the same way, and it is not the way people expect. They do not usually fail because someone made a calculation error. They fail because an assumption entered quietly, nobody wrote it down, and six months later a result that was true of a narrow case is being cited as though it were true generally. Nobody lied. The claim simply drifted, and no part of the system was watching the boundary.

I have spent several years building a research program in foundational physics. What has been published out of it so far is mathematics, not physics, and the repository says so on its front page: the physical program is ongoing and is not claimed. Whether that larger program survives contact with people qualified to break it is now a question I do not control, which is exactly where I wanted it. What I want to write about here is the other thing the program produced, which is separable from the subject matter entirely: a governance system for doing work under real uncertainty, designed around the assumption that I am the most likely source of error in it.

I think that system is worth studying on its own, and none of it is specific to physics.

Commit before you know

The single rule that does the most work in the whole program is that positions are filed before the work runs.

Before a research round begins, I write down what I expect to happen and what result would prove me wrong. That document is dated, fingerprinted, and frozen. When the results arrive, they are scored against what was written, and reinterpretation after the fact is not permitted. If the result is ambiguous under the wording I chose, the honest outcome is that my stake was badly worded, and it is recorded as undecided rather than quietly resolved in my favor.

This feels unnecessary right up until the first time it saves you. Human memory of what we expected is reconstructed after the fact and is reliably generous. Writing the prediction down removes the reconstruction step.

There is a second layer to this that I would recommend to anyone working alongside an AI system or a strong analyst. When I work a round with a research instrument, I file my position first and only then does the instrument file its own, independently. The reason is that if I see a confident position before forming mine, I will anchor to it without noticing. Ordering the two disclosures correctly costs nothing and preserves the independence of two judgments instead of collapsing them into one.

Records that only grow

Every ledger in the program is append only. Corrections are appended; the original text is never edited.

When a locked result is later found to contain an error, the fix is written as a dated addendum underneath it and the wrong sentence stays exactly where it was. This is uncomfortable. It also means the record shows not only what I currently believe but what I used to believe, when it changed, and what changed it. A record that silently updates cannot tell you whether the program is learning or just drifting.

This is the most commonly skipped discipline and the one I would give up last.

A test can be void

A test in this program is valid only if it was both written correctly and executed correctly.

That means a badly designed test that runs flawlessly is void. Its data is discarded rather than salvaged, and the objectives it was supposed to settle transfer to a successor test to be earned again. I have voided a test in my own program under this rule after it had already produced a result I wanted, and the credit it appeared to earn was struck along with everything else.

The reason to state the rule in advance is that it removes the incentive to rescue a broken experiment by reinterpreting its output. Once you have data, the pull toward finding something usable in it is very strong, and that pull is where most self deception in technical work actually lives.

Verification by someone who cannot see your reasoning

Before publishing anything externally, the work goes to a reviewer that has no access to my working context, no knowledge of what I hoped to find, and explicit instructions to attack the claims rather than confirm them.

The most recent pass caught a real error. One of the theorems described a geometric region incorrectly. Every formula and every count in the theorem was right, but the description of the shape was wrong, and it had survived my review because I had been reading it in a coordinate system I never mentioned I was using. That is precisely the kind of error a reviewer who shares your context cannot see, because they are standing where you are standing.

The full review is published alongside the results, unedited, including the part where it caught me. I would rather ship the record of being corrected than a clean surface with a hole under it.

Keep the dead ends

Conjectures that turned out to be false are kept in the record with the mechanism of failure named.

At one point I expected a certain class of structure to be impossible. It is not; counterexamples exist, and I built them. Rather than delete the wrong expectation, it is recorded as a dead end, and the counterexamples became a published section of a paper, because knowing precisely why a natural argument fails is worth more than the argument would have been if it had worked.

Failure mechanisms are reusable. Successes usually are not.

State what you actually searched

Mid program, a review turned up something that changed the rules going forward. A claim of the form “no valid alternative exists” had been supported by a search that only explored a restricted set of alternatives. Nobody had misrepresented anything. The search was real and the result was real. But the sentence describing it was broader than the evidence underneath it.

The rule that came out of that is now permanent: every claim that nothing satisfies some condition has to state, on its face, exactly what space was explored and whether the exploration was complete. The earlier claim was not deleted. It was narrowed of record, with a dated note explaining why.

A governance system proves itself when it catches its own author, and the correct response to being caught is a new rule rather than an apology.

Why this belongs on a business site

I do systems architecture. When someone hires a technical architect, the real question is almost never whether that person is capable. It is whether their process makes their own errors visible to them before those errors reach production.

Everything above is a direct answer to that question. Part of it you can check today. The external review that found an error in my own theorem is published unedited in the research repository, including the section where it caught me, and I did not get to choose which parts of that document to show. The correction it forced is in the release notes. A falsified expectation of mine sits in the measurement log.

The rest of what I have described lives in ledgers that are not yet public, which means that for now you have my account of them rather than the records themselves. I would rather say that plainly than imply an auditability I have not provided. Those records go out as the research program’s own materials reach release quality.

The same structure applies to any program where the cost of a quiet wrong assumption compounds: infrastructure migrations, platform rebuilds, data pipelines, anything where a decision made in month one is load bearing in month nine and nobody remembers making it.

What it costs

It is slower. That is not a rhetorical concession, it is a real cost. You spend hours writing down predictions that turn out to be wrong. You maintain records that mostly nobody reads. You occasionally throw away work that was executed perfectly because the thing it was executing was badly specified.

What you buy is that when you are wrong, you find out from your own system rather than from a customer, a reviewer, or a public correction. On the schedule that matters, that is not slower at all.

Where this stands

The mathematics is published, with verification code and a claim by claim map pairing every claim to the check that verifies it and to its evidence strength, stated individually as proof, computation, witness, or open. Four problems I could not solve are published as open rather than deferred, and one of them is posed formally as an open problem for anyone who wants to take it. It is open to attack, and the invitation is real. Negative results will stay in the record with credit to whoever finds them.

Current status, the repository, the release, the open problems, and how to report an error are all on the program page: Ball-Lens: Order and Counting on Discrete Substrates. That page is kept current. This one is dated.

Whether the physics survives is genuinely unknown to me. The method has already earned its keep.