Short answer
DRACO (IBM, arXiv:2609.04094, code at github.com/IBM/draco) attacks the credit assignment problem in long-horizon agent learning: when a trajectory succeeds or fails, deciding which steps were responsible is hard, and most training pipelines smear credit across everything.
Its approach: generate a rubric specific to each trajectory and prompt, remove redundant rubric items, have a judge cite which steps each rubric item depends on, and redistribute credit through those citations - all without training a separate attribution network.
Ernesta Labs read it as a way to make agent learning auditable: credit is attached to cited steps, so you can inspect why a change was promoted. The paper reports results on AppWorld and Tau-Bench, but we treat the exact numbers cautiously - the paper and repository README are not perfectly aligned, and we take the more conservative reading everywhere they disagree.
The failure mode: successful runs that teach you nothing
Every agent team that tries to 'learn from its trajectories' runs into the same wall. A 180-step run succeeds. Was it the good retrieval step at step 12, the lucky retry at step 97, or just a forgiving environment? If you fine-tune on the whole run, you reinforce all of it - the good, the bad, and the indifferent. Next month your agent is confidently doing the lucky retry.
This is credit assignment, and it is the quiet killer of agent improvement programs. It is not a modeling problem you can pay off with scale. It is a bookkeeping problem: which steps in this trajectory earned the outcome, and can a human check the answer? Most pipelines answer 'all of them, roughly' and hope.
The second failure mode is worse: attribution networks. If you train a separate model to assign credit, you have moved the trust problem, not solved it. Now you cannot say why a step got credit - the attribution model said so, and its reasons are not inspectable.
What DRACO actually does
PRIMARY SOURCE RESULT: DRACO (IBM, arXiv:2609.04094, code at github.com/IBM/draco) proposes dynamic, trajectory- and prompt-specific rubrics for credit assignment in long-horizon agents. Instead of one static scoring rubric applied to every run, DRACO decomposes each trajectory into rubric items generated for that specific prompt and run.
PRIMARY SOURCE RESULT: The pipeline then removes redundant rubric items, so overlapping criteria do not double-count credit. A judge model evaluates the run against each remaining rubric item and - this is the part Ernesta Labs cares most about - cites the specific trajectory steps each rubric item depends on. Credit is then redistributed across those cited steps rather than spread uniformly across the whole trajectory.
PRIMARY SOURCE RESULT: The paper reports evaluation on long-horizon agent benchmarks including AppWorld and Tau-Bench. A deliberate design property: no separately trained attribution network is required. Credit comes from the judge's citations against an inspectable rubric, so the attribution is traceable to text a human can read.
What they found, and how much we believe it
PRIMARY SOURCE RESULT: On AppWorld and Tau-Bench, the paper reports that rubric-based credit assignment improves agent learning compared with uniform credit assignment, with the rubric decomposition and redundancy removal contributing to the gains.
We want to be honest about the caveats before anyone copies a number. First, AppWorld and Tau-Bench are simulated, well-instrumented environments. They have ground-truth success signals and scriptable tasks. Your production agent does not. Second, the paper's reported effect sizes should be read as 'the mechanism helps in these settings', not 'attach this and your agent gets better'. Third, the repository README and the paper do not present identical claims; where they diverge - in framing, in what is emphasized, in how results are summarized - we take the weaker, paper-conservative reading. If a claim only appears in the README, we treat it as unverified by paper.
None of this makes DRACO wrong. It makes DRACO research-grade evidence: a mechanism, demonstrated in benchmarks, with code. That is genuinely more than most 'our agent learns from experience' pitches can show.
Limitations, stated plainly
The biggest one, and DRACO's authors would not dispute it: rubric attribution is not commercial causality. A judge citing steps 12 and 97 against a rubric is a structured, inspectable opinion. It is not proof that changing step 12 would have changed the sales outcome, the deployment, or the customer decision. In domains where outcomes arrive days later and depend on other humans, rubric citations get you 'plausible locus of responsibility', not causal effect.
The judge itself is the epistemic weak point. The rubric is generated by a model, the citations are made by a model, and both inherit the judge's biases. Redundancy removal helps but does not eliminate rubric drift: a generated rubric can quietly omit the criterion that actually mattered. And a single-paper, benchmark-scale evaluation does not establish how this behaves under distribution shift or adversarial trajectories.
Study scale also matters. This is one paper, two benchmark families, one code release. That is enough to justify a builder prototype. It is not enough to justify wiring it into a production promotion loop unattended.
Why builders should care
Because every team building a serious agent eventually wants it to improve from its own runs, and almost everyone ships the same broken version: 'the run succeeded, log the whole transcript as a good example, retrieve it later'. That is uniform credit assignment with extra steps. It pollutes few-shot libraries with luck.
The alternative DRACO gestures at is something auditors and regulators can actually work with: before a trajectory becomes learning evidence, someone - a judge, a human, both - decomposes it into criteria, cites which steps mattered, and signs the result. Promotion decisions then reference that artifact. 'We promoted this skill change because rubric items R3 and R7 passed, citing steps 12 and 97' is an auditable sentence. 'The agent learned from the run' is not.
If you cannot produce the first kind of sentence for the changes your agent makes to itself, your agent does not have a learning system. It has a habit-forming system.
Ernesta Labs interpretation
LABS INTERPRETATION: DRACO's real contribution is not the rubric format - it is making credit assignment a first-class, inspectable artifact. The judge-cites-steps mechanism converts an opaque training signal into a chain of evidence you can audit, dispute, and reject. That is the property long-horizon systems need, because they accumulate self-modifications and you must be able to reverse the reasoning later.
LABS INTERPRETATION: We also read the no-attribution-network design as a statement of values: attribution you cannot explain is attribution you cannot trust. We agree. A learned attribution model is faster and probably more accurate in-distribution, but it is unauditable, and unauditable credit is how self-improving systems drift.
Where we differ from the paper's implied frame: we do not treat rubric citations as sufficient grounds for automated promotion. In our model, cited steps make a trajectory eligible to become learning evidence; eligibility is not promotion. Promotion requires a separate, conservative gate (Article 30 and Article 31 develop this).
What we would implement
LABS RECOMMENDATION: Implement the pipeline as a promotion-preparation stage, not a promotion authority: (1) verified trajectory in, outcome attached; (2) dynamic rubric decomposition - rubric generated per trajectory and prompt; (3) redundant-rubric removal; (4) judge evaluates the trajectory against each rubric item and cites the relevant steps; (5) causal attribution marked explicitly as UNKNOWN - the citations are loci, not causes; (6) only then does the trajectory become eligible learning evidence; (7) any candidate change derived from it must be localized to the cited steps and validated separately before promotion.
LABS RECOMMENDATION: Keep every intermediate artifact - the generated rubric, the redundancy-removal decisions, the citations - stored alongside the trajectory. The pipeline's value collapses if the citations are not preserved for later inspection, dispute, and rollback analysis.
For stack decisions: run it with a strong judge model you can swap, evaluate a sample of rubrics by hand before trusting the generator, and log rubric drift metrics over time - rubric item overlap, citation concentration, and how often citations point at steps a human reviewer disputes.
What we would not implement
We would not let DRACO-style attribution trigger automated self-modification in production. The paper does not claim commercial causal validity, and neither do we. Rubric citations rank candidates; they do not authorize changes.
We would not use it where outcomes are ambiguous or delayed without a human closing the loop. On a sales trajectory where the customer replies four days later, rubric citations about the trajectory tell you what the agent did, not what moved the human. Treating those as equivalent is exactly the kind of self-deception this arc exists to prevent.
And we would not skip the redundancy-removal step to save tokens. Double-counted rubric items are precisely how a judge inflates credit onto one lucky step.
CASE STUDY - NIKO: credit assignment before skill changes exist
Ernesta Labs has not yet wired DRACO-style attribution into NIKO's runtime, and we are explicit about that: today NIKO's evidence ledger records actions, reality-verification results, and outcomes, but no rubric-based decomposition of trajectories. The ledger is the prerequisite. If you cannot reconstruct the trajectory, you cannot cite steps in it.
The near-term plan follows this article's synthesis: once a body of verified trajectories exists, generate dynamic rubrics over them, have a judge cite steps, and mark causal attribution UNKNOWN. That output feeds the candidate-skill pipeline described in Articles 26 and 28 - a localized, versioned candidate, never an in-place mutation. A trajectory would only become 'eligible learning evidence' after that decomposition, and promotion stays a separate human-supervised gate.
What we will not do: let a judge's citations edit NIKO's skills directly, or treat a successful outreach sequence as proven sales technique because a rubric said the copy step was cited. A reply is not a meeting; a meeting is not a customer.
DESIGNED: rubric-based eligibility for learning evidence. IMPLEMENTED: the evidence ledger and trajectory capture it depends on. NOT YET PROVEN: that rubric-derived eligibility improves any NIKO outcome. TESTING has not begun.
Practical test: where does your credit actually land?
Run this today on your own agent, no new infrastructure. Take five recent successful trajectories and five failures. For each, write down which specific steps you believe were responsible for the outcome, and force yourself to cite step numbers. Then hand the same transcripts to a strong model with this prompt: 'Decompose this trajectory into 3-6 criteria that determined the outcome, remove redundant criteria, and cite which steps each criterion depends on.'
Now compare. Two things will surface fast. First, your own citations and the judge's citations will disagree on some runs - every disagreement is a finding about how little you actually know about your own system's success. Second, count how many steps received credit. If the honest answer is 'most of them', you have uniform credit assignment: your learning signal, if any, is reinforcing luck.
Then the decisive check: for your most-recently-improved prompt, skill, or workflow, ask what evidence promoted it. If the answer is a feeling, a single good demo run, or 'the whole transcript was good', you have found the work. DRACO's pipeline is one auditable way to build it; the point is that a chain from outcome to cited steps to localized change must exist and be inspectable.
What remains unknown
- Whether judge-cited rubric attribution transfers from benchmark environments to commercial domains with delayed, human-dependent outcomes - DRACO does not test this and we do not assume it.
- How much rubric generation itself drifts: whether generated rubrics systematically omit the criteria that mattered, and how to detect that omission without ground truth.
- Whether redundancy removal helps or harms attribution accuracy on trajectories where multiple genuinely distinct criteria overlap in wording.
- The paper/README discrepancies noted above: which claims replicate, and at what scale, remains unverified beyond the reported AppWorld and Tau-Bench evaluations.