Research note / External audits
Auditing self-improving AI systems
27 issues filed in 23 open-source self-improving AI projects, looking for the places where a system can approve its own unproven gains.
An issue is a report or a proposal on someone else’s repository, not a confirmed defect. Where another person reproduced the problem or implemented the fix, that response is linked. Most issues are still open, and none of this is peer review.
When the system grades its own work
A self-improving system proposes a change, tests it, and keeps it if the test says it helped. If the same system also writes, tunes or selects the test, a change can pass without being real: the evaluator is gamed, the held-out data leaks into the search, or the selection step keeps the wrong candidate. Nothing in the reported score shows the difference.
This matters more as AI does more of its own development. In September 2026 Anthropic described an internal R&D automation index in which Claude agents gather the evidence and a separate Claude judge assigns each automation level (report). The question this record asks is narrower and testable today: in open-source self-improvement code, where can an unproven gain get through, and does anyone else see the same problem?
For the concept and the controls a claim needs, see the guide to recursive self-improvement.
What others reproduced or implemented
Five of the issues drew a concrete public response from someone other than me: a reproduction, a counterexample, or code.
Skill evolution was a no-op
In NousResearch’s hermes-agent-self-evolution, the evolved skill text was the unchanged baseline, so the optimizer never touched it. Two other users reproduced this on different stacks, one with a diff showing the evolved and baseline files byte-identical; a third reported a fix.
Issue #141 →A gate that accepts evaluator gaming
In Microsoft’s SkillOpt, another user later reported a reproducible case: the edit gate accepted a rule that gamed a heading check, lifting the validation mean from 0.682 to 0.852 while one task regressed. The gate compared means with no per-task regression check.
Issue #67 →Negative-control probes
A collaborator on EvoHarness implemented the proposal: a candidate that passes replay and regression can still be blocked when a near-miss probe shows it claiming success where it should fail.
Commit b8f323e →An admission ledger for GEPA
An external contributor implemented a per-decision record of every accepted and rejected candidate, so an acceptance can be replayed. I reviewed it; it was open on the check date.
Pull request #382 →Darwin Gödel Machine selection
The “best” parent-selection mode could not be chosen from the command line, and when called it sorted toward the lowest scores. Another contributor opened a fix; it was open on the check date.
Pull request #32 →Seven recurring failure patterns
The 26 active issues group into seven ways a self-improvement loop can report progress it has not shown. Each pattern lists the issues that raise it.
- The optimizer never touches what it claims to optimize. The evolved artifact is byte-for-byte the baseline, so any reported gain came from somewhere else. NousResearch/hermes-agent-self-evolution#141
- Selection picks the wrong end. A mode meant to keep the best candidates cannot be chosen, and when called it keeps the worst. jennyzzt/dgm#31
- A mean-score gate accepts a regression. An edit that games the evaluator lifts the average while a task gets worse, and the gate only looks at the average. microsoft/SkillOpt#67, microsoft/stop#7
- Held-out answers leak into the search. Memory retrieval, generated data or rubrics can expose the answer the system is later scored against. google-research/reasoning-bank#17, aiming-lab/Agent0#30, modelscope/AgentEvolver#48, OpenDataBox/Workspace-Bench#8, neosigmaai/auto-harness#25
- No record of what was admitted or rejected. Without a per-decision ledger, nobody can replay why a candidate was kept, so a lucky or contaminated acceptance is invisible. gepa-ai/gepa#381, facebookresearch/HyperAgents#35, Arvid-pku/Godel_Agent#4, facebookresearch/matrix#138, HolobiomicsLab/Mimosa-AI#154
- Gains promoted without a negative control. A change is promoted because it helped where it was tried, with no test that it does not also claim success where it should fail. iLearn-Lab/EvoHarness#1, VectorInstitute/helix#86, hexo-ai/sia#38, JARVIS-Xs/SE-Agent#516, Human-Agent-Society/CORAL#124, NousResearch/hermes-agent-self-evolution#118, NousResearch/hermes-agent-self-evolution#115
- Configuration that silently changes the run. Ignored timeouts, misrouted output paths and outdated setup make a reported run hard to reproduce. neosigmaai/auto-harness#24, kayba-ai/recursive-improve#2, microsoft/stop#6, zou-group/sirius#2, VectorInstitute/aieng-bot#105
The same patterns appear in this project’s own work: a 15/15 result that the task generator guaranteed, and a recursive chain whose reported gain reversed under shared random streams. Both are in the failure log.
The full record
Every issue filed in June 2026, with its state and any public response as of November 5, 2026. One issue was withdrawn by me shortly after filing and is listed for completeness.
| Project and issue | What it raises | State | Public response |
|---|---|---|---|
| NousResearch/hermes-agent-self-evolution#141 | Skill evolution is a no-op: optimized_module.skill_text is the unchanged baseline | Open | Reproduced independently by two other users; a third reported a fix |
| jennyzzt/dgm#31 | best parent-selection mode is unreachable from CLI and selects lowest scores when called | Open | Fix opened by another contributor as PR #32 (open) |
| microsoft/SkillOpt#67 | Add a verifier-discipline stress test for reward-hacking skill edits | Closed | Another user reported a reproducible counterexample after closure |
| microsoft/stop#7 | Require a relative utility gate before adopting the next self-improver | Open | — |
| google-research/reasoning-bank#17 | Add a leave-one-task-out guard for memory retrieval to prevent self-retrieval contamination | Open | — |
| aiming-lab/Agent0#30 | Add an admission manifest for generated-question data and answer-leakage canaries | Open | — |
| modelscope/AgentEvolver#48 | Add a canary audit for ground-truth/reference-solution isolation | Open | — |
| OpenDataBox/Workspace-Bench#8 | Add a canary leakage audit for rubric and ground-truth isolation | Closed | Discussion (3 comments) |
| neosigmaai/auto-harness#25 | Add sealed negative-control tasks to audit train/test leakage and gate discipline | Open | — |
| gepa-ai/gepa#381 | Add a run_dir admission manifest for candidate acceptance/rejection audits | Open | Implemented by an external contributor as PR #382 (open) |
| facebookresearch/HyperAgents#35 | Add machine-readable admission manifests for generated agents and rollback decisions | Open | Discussion (1 comment) |
| Arvid-pku/Godel_Agent#4 | Add a modification ledger before dumping accepted self-modified logic | Open | Discussion (1 comment) |
| facebookresearch/matrix#138 | Add optional acceptance/provenance gates for synthetic multi-agent outputs | Open | — |
| HolobiomicsLab/Mimosa-AI#154 | Add workflow admission cards before similarity-based template reuse | Open | Discussion (2 comments) |
| iLearn-Lab/EvoHarness#1 | Add negative-control probes to promotion validation | Open | Implemented by a repository collaborator (commit b8f323e) |
| VectorInstitute/helix#86 | [FEATURE] Add transfer-evidence cards before promoting Helix improvements to main | Open | — |
| hexo-ai/sia#38 | Add transfer-evidence cards to separate reusable improvements from task-specific residue | Open | — |
| JARVIS-Xs/SE-Agent#516 | Add counterfactual admission gates for trajectory-pool reuse | Open | — |
| Human-Agent-Society/CORAL#124 | Add quarantine-backed promotion gates for cross-island memory and skill migration | Open | Discussion (4 comments) |
| NousResearch/hermes-agent-self-evolution#118 | Phase 4 verifier/fitness harness: sealed hidden evals, rollback traces, and adaptive-vs-frozen gates | Open | — |
| NousResearch/hermes-agent-self-evolution#115 | [Design Proposal] Runtime-gated reingestion with negative-control probes for skill evolution | Closed | — |
| neosigmaai/auto-harness#24 | Terminal-Bench per_task_timeout config is ignored by benchmark and gate runners | Open | — |
| kayba-ai/recursive-improve#2 | evolve update writes scores to a nested island status file when run from a worktree | Open | — |
| microsoft/stop#6 | Add reproducible setup notes for current Python/OpenAI SDK environments | Open | — |
| zou-group/sirius#2 | Make feedback and finetune-data paths respect subject/model args | Open | — |
| VectorInstitute/aieng-bot#105 | Align agentic fix-loop staging instructions and result-comment step id | Closed | Discussion (2 comments) |
| SakanaAI/AI-Scientist#249 | Withdrawn by the author shortly after filing | Withdrawn | — |
Follow-up code: pull request #202 to hermes-agent-self-evolution adds lesson probes, a regression gate that checks each lesson by removing it and confirming the behavior it provided. It was open on the check date.
What this record does not show
- An open issue is a claim by its author. Only the five cases above have a public reproduction, counterexample or implementation by someone else.
- Maintainers have not merged most of the proposals, and a closed issue is not necessarily a resolved one.
- The issues were written with AI agents under my direction and filed from my account. The same verification standard applies to them as to the systems they audit.
- Responses are counted only when visible on the issue page; private replies and later changes are not included until the next check.