Short answer
Harness-of-Harness (arXiv:2609.01481, with code on GitHub) studies multi-day autonomous software development under continual, independent verification. The design separates a Planner, an Executor, and an independent QA harness, so the component that judges the work is never the component that produced it.
The core principle Ernesta Labs adopted: SELF-TEST IS NOT INDEPENDENT VERIFICATION. An agent grading its own output will reproduce its own blind spots with high fidelity. Separate task execution from independent evidence checking, keep the evidence persistent, and never let a failure be silently closed.
The failure mode: the worker is also the grader
Almost every agent stack ships with a self-test story: the agent writes the code, then the agent runs the tests, then the agent reports that the tests pass. For a five-minute task this is merely weak. For multi-day work it is structurally broken, and the reason is not capability but corruption of the evidence chain.
When one process produces work and evaluates that same work, its evaluation inherits every blind spot of the production process. If the agent misread the requirement, the same misreading shapes its idea of what 'passing' means. If the agent's change broke something in a module it never looked at, its own test selection will not go looking there either. The verdict 'done' from a self-grading agent is a claim, not evidence. Over a multi-day run, claims that are never checked against independent evidence compound: each unverified 'done' becomes a false premise for the next day's plan.
There is a second-order version of the problem that only shows up at multi-day scale. Self-test failures are not independent events; they correlate, because they share a cause - the same misunderstanding, the same broken assumption, the same drift in the world since the plan was written. When your only grader is the worker, correlated failures are invisible in exactly the runs where they cluster. The longer the run, the more of your confidence rests on the least-verified claims.
This is the failure mode Harness-of-Harness is built against, and it is the reason we treat the following sentence as a design law rather than a slogan: SELF-TEST IS NOT INDEPENDENT VERIFICATION.
What the paper actually does
PRIMARY SOURCE RESULT: Harness-of-Harness (arXiv:2609.01481, code at github.com/Flesymeb/HarnessOfHarness) proposes and evaluates a supervision architecture for multi-day autonomous software development. The design separates three roles: a Planner that decomposes and schedules the work, an Executor that performs task work, and an independent QA harness that continually checks the executor's output against the objectives. Verification is continual rather than terminal: the QA harness re-examines the running system throughout the multi-day process, not only at the end.
PRIMARY SOURCE RESULT: The architecture maintains persistent evidence bundles - structured records of what was done, what was checked, and what the checks found. Two properties of these bundles matter most to builders. First, unresolved gaps are kept as unresolved: the system does not allow open issues to be silently dropped from attention. Second, failures can be reopened: a previously 'resolved' finding that later evidence contradicts returns to the open set instead of being buried under newer work.
PRIMARY SOURCE RESULT: The reported work spans runs of substantial length, with individual pieces of work persisting across 70+ iterations under supervision. The harness also uses progressive disclosure: verification capabilities and context are exposed to the supervised work in layers, rather than dumping the full harness surface at once.
What they found, and why the separation is the point
The result worth remembering is not a number but a structural claim: when verification is independent, continual, and persistent, multi-day agent work stays honest in a way self-tested work does not. The independent QA harness does not need to be smarter than the executor. It needs to be differently positioned: it reads the objectives and the evidence, not the executor's self-narrative. A grader that never shares the executor's reasoning path cannot inherit the executor's rationalizations.
The persistent evidence bundle changes what 'done' means. In a self-test loop, done means the worker said so. In this design, done means there is an evidence record showing an independent check passed, and that record is still open to contradiction later. Reopened failures are the visible proof of that openness: a system that can never reopen a failure is a system whose history can only accumulate confirmations.
Progressive disclosure, meanwhile, addresses a quieter engineering problem: a supervision harness that floods the executor with every policy and check at once produces noise, and noise is what agents collapse into. Disclosing verification capabilities in layers keeps the supervised work pointed at the checks that matter for its current stage.
The 70+ iteration figure deserves a careful reading. Its value is not as a leaderboard number but as evidence of the timescale mismatch the architecture addresses: ordinary agent harnesses are designed around sessions measured in minutes, where a human can eyeball the transcript at the end. At seventy-plus iterations of continuing work, no human eyeballs anything in time. The supervision loop has to carry the load continuously, because there is no 'after' at which a human arrives before damage compounds. That is the difference between verification as a final gate and verification as an operating condition.
One more finding deserves emphasis because it is counterintuitive: continual checking caught problems that end-of-run checking missed. A failure introduced on day one and contradicted on day three can look 'resolved' by day five if nobody was watching on day three - the code changed again, the symptom moved, the history got complicated. Continual verification with reopen semantics keeps day-three contradiction alive as a first-class object until evidence, not elapsed time, retires it. Most teams do the opposite: the older a 'done' claim, the more unconditionally everyone trusts it.
Limitations, stated plainly
This is a study of autonomous software development - a domain with an unusually strong verification substrate: compilers, tests, and runnable systems give an independent harness objective signals to check against. Domains without such substrates (sales, research synthesis, relationship management) cannot import the architecture unchanged; they must first build the evidence layer that software gives you for free.
The results are reported by the paper's authors on their own harness and codebase, which is publicly available, but the evaluation is not a broad multi-team benchmark. The scale of the reported evaluation is modest by production standards, and we do not treat reported iteration counts as evidence of commercial reliability.
Finally, independence is only as real as the boundary. If the QA harness quietly shares state, prompts, or context with the executor, independence degrades to self-test with extra steps. The paper's design enforces the separation; an implementation that copies it structurally but not informationally gets nothing.
Why builders should care
If your agent runs longer than one session, you already have a verification debt problem: unverified 'done' claims accumulating somewhere in your run history. The only question is whether you have an independent process that can catch them, or whether the first thing that catches a false 'done' is your customer.
The builder-level consequence is concrete. Every long-running agent needs an answer to three questions: Who checks the work, and can that checker be influenced by the worker's own reasoning? Where does the evidence live after the run ends? And can a closed failure be reopened when new evidence contradicts it? If your answers are 'the agent itself', 'nowhere', and 'no', your multi-day system is running on claims.
Notice what this does to your evaluation habits, too. Most teams evaluate agents by final-output inspection: read the answer, judge the answer. That is exactly the self-test trap at the team level - the human grader sees the polished endpoint and none of the unverified 'done' claims buried in the middle of the run. Independent verification with persistent evidence makes the middle of the run inspectable, which is where multi-day failures actually live.
Ernesta Labs interpretation
LABS INTERPRETATION: We read Harness-of-Harness as the strongest available statement of a principle that most agent stacks still violate by default: judgment must be structurally independent from production. The Planner/Executor/QA separation is one instantiation of the principle in one domain; the principle itself - separate task execution from independent evidence checking - transfers everywhere.
We also read the persistent evidence bundle as the load-bearing component, more than the role separation. Roles can be simulated by one process wearing three hats; a persistent, contradiction-tolerant evidence record cannot be faked by a single-threaded loop. That emphasis shaped how we design NIKO's runtime and how we evaluate any harness we consider adopting.
What we would implement
LABS RECOMMENDATION: Implement the separation, not the domain. Concretely: (1) an executor that performs task work and appends every action and artifact to an evidence log; (2) an independent verifier with its own read path over objectives and evidence, never over the executor's internal narrative or self-assessments; (3) persistent evidence bundles that survive restarts, in which unresolved gaps remain first-class objects; (4) reopen semantics - any 'resolved' verdict is a claim against current evidence and can be reverted by newer evidence; (5) progressive disclosure of verification scope, so the executor sees the checks that matter for its current stage rather than the entire policy surface.
For commercial agents outside software, invest first in the evidence substrate: define, in writing, what independent evidence means in your domain before you build the verifier that consumes it.
What we would not implement
We would not implement a verifier that shares its context window, model session, or prompts with the executor - that is self-test wearing a costume. We would not implement 'reopen' as a human-only workflow; a reopened failure that requires a human to notice it is not a mechanism. And we would not port the software-specific machinery (compiler-style gates, test-suite loops) to non-software domains and call the result independent verification; without a domain evidence substrate, the QA harness is just a second opinion with better manners.
CASE STUDY - NIKO: one implementation of the separation
NIKO's runtime implements the principle this paper argues for: a Mother objective specifies what must be true, NIKO chooses and performs an action, and a separate reality-verification step checks the recorded world for evidence of what actually happened. The verification step reads evidence, not NIKO's self-report. To be precise about credit: the paper did not specify this design and NIKO did not copy it from the paper; NIKO's runtime is one independent implementation example of the same structural principle, reached in parallel and later validated against this source.
A signup that was recorded is not a customer. An email that was sent is not a sale. NIKO's evidence chain keeps those distinctions material, and unresolved gaps in NIKO's record stay unresolved until evidence closes them - never NIKO's confidence.
DESIGNED: independent verification layer with reopen semantics. IMPLEMENTED: Mother-objective separation and reality verification in the runtime. TESTING: whether the separation catches real contradictions during live operation. NOT YET PROVEN: that independent verification measurably improves NIKO's outcomes versus a self-tested baseline. Nothing in this article claims it does.
Practical test: does your agent verify itself?
Run this on your current system today. Pick three recent runs your agent marked 'done'. For each, answer: (1) Which component produced the work? (2) Which component judged it - and does that component share state, context, or prompts with the producer? (3) Where is the evidence that the check happened, stored independently of the run's own memory? (4) If tomorrow you learned the check was wrong, is there any mechanism that reopens the failure - or is it permanently closed in your records?
Scoring: if any of the four answers comes back as 'the same process', 'shared', 'nowhere', or 'no', you are running on self-test. The fix does not require a new model: create a second read path over your run's evidence with access to the original objectives and none of the executor's self-assessments, and let it write verdicts the executor cannot edit. That is the minimum viable independent verifier.
What remains unknown
- How well the Planner/Executor/QA separation transfers to domains without a runnable verification substrate like sales or research synthesis.
- Whether independent verification measurably improves long-run outcomes versus self-test, in a controlled comparison - the paper reports architecture, not that comparison.
- The cost profile: independent continual verification multiplies checking work, and the paper does not establish the economic trade-off for commercial deployments.
- How quickly independent verification degrades when the verifier and executor share any context at all - the boundary conditions are not quantified.