Making an Agent Improve Itself — Five Months of Actually Trying

I did not want to explain the same mistake to an agent twice. If we had fixed something, I wanted the next task to benefit. Frequently used procedures should become skills. A failure should leave the agent a little less likely to get lost next time. But if I had to request each of these changes myself, calling it self-improvement would be a stretch.

I also did not want the agent to distribute whatever it happened to consider a good idea. Put an incorrect interpretation into memory and it becomes a premise for the next task. Distribute a broken skill and the mistake spreads with it. What I wanted was improvement without repeated instructions, but no unverified changes on the agent's own authority.

The work started in May 2026 by extracting useful lessons from a public self-improvement skill. In July, we reopened the route from ordinary work to skill candidates. August brought fixed-task comparisons. In September, we found functionality missing from the migration to Hermes. October was spent fixing the remaining gates between evaluation and deployment. This is an account of those roughly five months. The latest observation covered here is from the morning of October 15; it is not a statement about operations after that point.

One distinction matters throughout. We changed skills, operating instructions, and a few management tools, not model weights. This was not an experiment in training the underlying model through reinforcement learning. The account comes from preserved Wiki notes, work logs, and the evaluation, approval, and deployment receipts linked in those records. We did not rerun operational tasks to write the article. I distinguish what the records establish from my interpretation of them.

We did not begin by adding another memory system

The public self-improving skill we examined in May had useful ideas: learn from user corrections, retain solutions to repeated failures, and distinguish lasting rules from temporary notes. Installing it unchanged, however, would have added another memory hierarchy alongside the existing memory and Wiki. We would first have needed to decide which one was authoritative.

Instead, we absorbed the lessons into the existing system. Explicit corrections, stable user requirements, and repeated failures with clear fixes could enter memory, the Wiki, or existing skills. Silence was not treated as success, and one ambiguous event was not promoted into a global rule. The contemporary record documents that choice and checks of skill distribution and visibility. It does not measure how much future work improved.

That distinction kept returning. A well-written lesson and a better next task are two different things.

An enabled job was not necessarily learning anything

At the end of July, we checked whether frequently used procedures were becoming skill candidates. The result was disappointing. A job named “Skill candidate harvest” was enabled, but its implementation checked required files and Wiki health. It did not inspect recurring session patterns or generate candidates. The weekly job that judged candidates was disabled.

The upstream update route was still running. But absorbing changes to an external skill is not the same as discovering lessons in my own daily work. Existing candidate documents did not prove a functioning collection route either.

The repair reused the existing structure. A lightweight sensor gathered bounded signals from recent material. Judgment ran only when signals or unresolved candidates required it, and the number of candidates handled at once was limited. Read-only trials and registration checks passed. That repair did not itself deploy a new skill. The records distinguish establishing a route from completing a promotion.

My first lesson was simple: an automation's name and a green scheduler status are insufficient. We need to examine what it actually reads and produces.

Compare on the same tasks before changing anything

In August, we reviewed Prime Agent's self-improvement design. Restricting edit types, rereading state before application, and retaining change history were useful patterns. We did not install its runtime alongside ours. It overlapped substantially with existing memory, skills, and task management, and its public self-improvement implementation mainly edited external state supplied to subsequent model calls. It did not provide our desired fixed-task before-and-after comparison as-is.

We chose this sequence:

Collect a signal → create one candidate → compare the current version and candidate on fixed tasks → apply through the owner → observe subsequent work.

Signals were limited to explicit corrections, verified failures, and repeated successes backed by distinct evidence. Candidates stayed inactive. We froze evaluation material and the pre-change state, then ran both versions under the same conditions. A central server's owner handled final application and distribution of shared skills. “Owner” means the existing execution route with authority and responsibility; it does not mean a person clicking an approval button for every iteration.

The early application records include a bounded evaluation of a real user correction that improved from 0.2 for the baseline to 1.0 for the candidate, followed by project-scoped application. Those scores belong to that evaluation pack. They do not mean the agent's overall accuracy rose from 20% to 100%, or that unrelated tasks would show the same benefit. Nevertheless, there was now something more concrete than “we made a good skill.”

The Hermes migration preserved deployment but lost collection

September's migration records expose another gap. Distribution and some Wiki and memory collection survived, but the collector that admitted user corrections and task failures was not connected to Hermes session and job data. Checks that services were running and skill files were visible did not catch the missing function.

Hermes' own background learning was a separate route. A locally created skill had not necessarily passed central evidence admission, evaluation, approval, or distribution. Even if managed imports could not be patched directly, a new overlapping local skill could still influence behavior.

We therefore treated Hermes as a candidate producer while retaining central authority over shared-skill activation. Corrections, failures, and learning candidates from individual agents would enter frozen evaluation, and the existing owner would apply passing changes. The request in early October reinforced this direction: I should not need to issue another instruction for every improvement, but evaluation must not be skipped.

October exposed the gates one by one

The following days were spent opening the actual path from evaluation to deployment. An external driver agent implemented candidate changes; the central owner reviewed and applied them. People were involved in repairing that infrastructure. This must be distinguished from the later records showing specific candidates being generated, evaluated, and deployed unattended.

The repair bundles progressed from v20 to v42. Listing every version would say little beyond “we changed many things.” The obstacles fall into four groups.

The first was deciding what to process again. Unchanged events repeatedly woke the model, legitimate rejections were classified as errors, and evaluated items remained open in the ledger. A cap intended to reduce repetition also parked work that was making progress. The answer was not simply to run more often, but to distinguish progress, rejection, out-of-scope work, and waiting, and continue each appropriately.

The second was a mismatch between evaluation contracts and evaluator outputs. A candidate could pass a behavioral comparison and then fail because the formal evaluation was asked for a metric it did not produce. Behavioral scores had been mixed with formal metric and check names. The fix was not to invent the missing metrics: it was to distinguish what each evaluator could report and reject invalid contracts both at admission and immediately before execution.

The third was a mismatch between the intended candidate and the code actually loaded. An outer approval-history check recognized a new evaluator, while an inner loader still compared only the old hash. Restaging once packaged an old draft in place of its replacement. Some proof code tested the installed version instead of the candidate. The existence of a function did not establish that the production call reached it.

The fourth was the final authority, deployment, and rollback connection. A management-tool candidate passed evaluation but had no application handler, so it stopped at a prepared submission. Cache-file policy changes conflicted with existing tests. After a failed deployment, local and remote release pointers could refer to different generations. One initial state replacement accepted a failed generation that the final reread then rejected. What survived an intermediate failure was harder than making individual functions pass.

Matching hashes do not establish correct behavior

One striking detail in the driver log is that held bundles had already passed extensive hash checks, integrity checks, and shell syntax validation. Checking 18,133 pinned entries did not eliminate incorrect policies or missing calls.

A hash tells us whether these are the approved bytes. It does not tell us whether those bytes behave correctly. Syntax validation similarly tells us that a statement can execute, not that it should.

The owner's holds caught real defects: a timeout rule exceeding the terminal's maximum, a guard disabled by default, a guard missing from the actual admission function, and another execution route bypassing it immediately before evaluation. For management-tool application, the owner also held a candidate that synthesized the actor's identity from environment variables and a rollback that could overwrite a later change. During the cache-policy change, the full suite exposed an existing invariant missed by a partial test run.

The proof method changed accordingly. Tested files and shipped files must match. Only the bundled candidate should be loaded, in the same way the central server loads it. Invalid input must fail at the actual admission and execution entry points. A sentinel at the next execution boundary can establish that rejected input never reached the runner. Valid input must still pass. The owner must be able to rerun the proof without folders that exist only on the producer's machine.

I would not read this as “run more validation.” My interpretation is move the checks to the place that can catch the actual failure. Repeating unrelated checks did not move the automation forward.

Two kinds of improvement actually reached deployment

The first concrete example was a CSS-checking change in frontend-taste. An earlier candidate improved behavioral results but was rejected over formal metric naming, with a separate risk involving removal of ignore markers. After the naming problem was repaired, a one-time successor evaluation was admitted, and a new compatibility-preserving candidate was evaluated. Establishing that readmission route was infrastructure repair.

The subsequent candidate generation, evaluation, approval, and deployment followed the automatic route. The driver log reports nine frozen CSS-checking cases, with training evaluation improving from 0 to 1, held-out evaluation from 0.625 to 1, and critical failures from 3 to 0. Formal evaluation also passed. A scheduled owner execution retrieved and applied the candidate. The deployment receipt records completion and a clean independent second audit; notification and ledger acceptance followed. The work record explicitly says nobody substituted for the automated approval and deployment stages of that candidate.

This establishes improvement on those CSS cases and completed deployment, not better frontend work in general. A later change to independent-success evidence URI admission in agent-refinement itself was also recorded as completed through the scheduled owner. No equivalent quality measurements are supplied for that second skill change, so deployment is the claim we can make.

The other kind of example was an alert improvement in cron_command_bridge.py. It began with a real failure on the night of October 14. During a network and DNS disruption, receiving a deployment artifact over FTP timed out. The evaluation job used that verified failure to propose that the existing alert did not explain the FTP response timeout specifically enough.

The candidate was small: seven added lines returning “Artifact FTP response wait timed out” only when a TimeoutError originated within the FTP function. Its proposal explicitly left retries, notification transport, and credentials unchanged, and made no claim that deployment or recovery had succeeded. It did not repair the network outage; it made one consequence more accurately identifiable.

Behavioral and formal evaluation accepted the candidate, but it initially stopped because there was no management-tool application route. After owner-reviewed repairs established that handler and the deployment and rollback linkage, a scheduled run applied it on the morning of October 15. The log summarizing the 06:44 status receipt records application at 05:06, installation readback, and ledger acceptance. The supported conclusion is that an agent proposed a small tool improvement from a failure and it was automatically applied through the authorized owner route. Whether users diagnosed incidents faster after seeing the alert was not measured.

This is not yet an agent that gets better every day

These examples matter because candidates did not remain documents: scheduled execution could carry them into application. They do not establish that every requirement from the five-month effort is complete.

The management-tool auto-deployment allowlist still contains only cron_command_bridge.py. Evaluators, integrity checks, owner handling, and audit code are not freely editable by the agent. Expanding the allowlist requires separate owner review and application. Watcher rules changed without approval and later reverted must reenter as formal proposals if needed.

The recovery records distinguish installation checks, isolated tests, Wiki cleanup, and actual business execution at different times. Earlier unresolved states must not be erased by later success logs. Later successes must be read within their transaction scope. A new success does not establish that every historical fault or every machine is repaired.

Text checks establish textual conditions. Deployment checks establish files and application routes. A model's self-assessment records its judgment. None alone establishes better work. Even gains on frozen tasks do not automatically prove generalization, fewer long-term failures, or lower total cost.

The substantial human and external-driver involvement in repairing the infrastructure also remains part of the record. People repaired the route; some candidates then deployed automatically along it. That is the boundary of what I can claim.

Five months ago, I thought retaining useful lessons would make subsequent work improve. Now I think the connection must extend through candidate creation, comparable evaluation, authorized application, and observation in later work. Leave out one stage and there can be a considerable distance between “self-improvement is enabled” and anything actually changing.

We reduced part of that distance this time. It is too early to promise continuous improvement. But there is now a record that lets us distinguish what ran automatically from what we still do not know.

Source scope

This article draws on the May 2026 pattern-absorption record; July's organic-promotion diagnosis and repair; August's Prime Agent review and bounded evaluation; September's Hermes learning and central-governance migration design; and October's unattended-operation request and recovery records. In particular, the external driver's original October 11–15 work log and October 15 retrospective were compared with central-server recovery notes from the same period. Internal execution addresses and identifiers are omitted. Reported gains retain their original evaluation scope.