Skip to main content

Learning

mecha learns how you want work done from the moments you stepped in. The signal is already in the transcripts — a mid-run steer, a denied tool call, a corrective follow-up turn are all recorded, so nothing new had to be captured to start.

The cycle is four commands:

mecha reflect # mine transcripts for interventions → one lesson each
mecha validate # measure whether the current rules change an answer
mecha learn # consolidate reflections into a rule set
mecha rules # what each rule has measured, and what should retire

Everything lives in files under ~/.mecha/learning/, which is a git repository. git log is the learning history; git revert is the undo.

PathWhat it is
reflections.jsonlAppend-only evidence, each pointing at its transcript
rules/<domain>.user.tomlYours. Never written by code, only read
rules/<domain>.learned.tomlConsolidation's output — edit or delete freely
validations.jsonlEvery attempted probe outcome, keyed to the rule set measured
validation-attempts.jsonlInput identities, reasons and arm receipts used to schedule retries
runs.jsonlOne audit record per consolidation pass
mined.jsonl, mined_outbox.jsonl, distilled.jsonlIdempotence ledgers
proposals/<id>.jsonRule changes waiting for a human

mecha reflect — interventions become reflections

Extraction from transcripts is pure code. Three kinds of intervention are recognised:

  • Steer — user text riding in the same message as tool results. Unambiguous: the user reached in mid-run to redirect.
  • Denial — a tool result reading Denied by the user: …. A recorded rejected intent. (A hook denial reads Blocked by a hook: instead, and is deliberately not mined — machine policy is not a user correction.)
  • Follow-up turn — a later user turn may be a correction or just the next task. Extraction only flags the candidate; the reflector decides, and is told to skip freely.

Each candidate goes to one model call, which returns the reusable lesson behind it. The result is appended to reflections.jsonl with the session id that proves it.

mecha reflect --dry-run # what would be mined, no model call, no writes
mecha reflect --limit 5
mecha reflect --sessions-dir /path/to/transcripts

A session whose reflections fail — a provider being down, usually — is left unmined for a later run to retry rather than marked and silently lost. Every writing pass takes the store's writer lock before reading what has been mined, so two concurrent closes cannot mine the same session twice.

mecha reflect also mines the outbox: an item that was sent with edits yields a writing-domain reflection from diff(staged, sent).

Reading the lessons before they are consolidated

mecha reflections # newest first
mecha reflections show 20260828T0915 # what happened, what was said, the lesson
mecha reflections edit 20260828T0915 # rewrite it in your own words
mecha reflections drop 20260828T0915 --reason "specific to one thread"

The store had no reader for a long time, and that is the wrong end of the pipeline to be blind at. A rule is a consolidation of several lessons, so by the time a proposal is reviewable the thing you wanted to disagree with has already been merged with four others and rewritten. The lesson is where a disagreement is cheap and precise.

edit is a provenance promotion rather than a text change: a lesson you typed yourself skips the model that would otherwise have laundered third-party bytes into it, which is the way an excluded reflection gets rescued rather than merely lamented. drop is a flag and never a deletion, on the same rule retired rules follow — a store that forgets its refusals offers the same lesson again next pass with nothing to say it was already judged; restore undoes it.

Nothing in these verbs calls a model or touches the network, and every write takes the store lock, so they are safe against a store the nightly is also using. The /learning modal in the TUI drives exactly them, alongside mecha rules list --json, and so does the web settings page — the same two panes, the same verbs as child processes, so no surface can do something to the store that the command line cannot.

mecha learn — reflections become rules

Consolidation groups reflections by domain and situation, then rewrites the rules for that region while preserving rules outside it. It absorbs new lessons, merges overlapping rules, and resolves contradictions. The stored rules/<domain>.learned.toml remains bounded across all regions.

mecha learn # apply immediately
mecha learn --min 5 # need this many unprocessed reflections (default 3)
mecha learn --holdout 0.25 # leave every k-th out, for validate to probe
mecha learn --auto # measure, then apply or refuse
mecha learn --propose # measure, then stage for owner review
mecha learn --dry-run

Rules ride in the system prompt under a ## Learned rules heading, user rules first, then matching learned ones, inside the cached prefix. The selected block is stable during a run and can benefit from prompt caching. --no-learned-rules opts out anywhere, and mecha eval forces it off so a scorecard measures the model rather than your accumulated rules.

The always-loaded block has two ceilings. RULES_CHAR_BUDGET (2600 characters) is the size half; MAX_ACTIVE_RULES_PER_DOMAIN (25 active learned rules per domain) is the count half, and it is a check that does not depend on the model listening to the frame instruction that says the same thing — the frame is handed the same constant, so the two cannot drift apart.

It is also per domain, and a run carries only the domains it asks for (RUN_DOMAINS: behavior and writing). A domain is opt-in, so a new one joins no prompt until something names it; a domain holding active rules that nothing carries is reported at startup, because rules that cannot fire look exactly like rules being obeyed.

"Routed" has two meanings, and the warning needs both. RUN_DOMAINS is what an agent run carries in its prompt. PASS_DOMAINS is what a named, tool-less pass loads — today just triage, which the mail classifier loads and which is deliberately absent from RUN_DOMAINS. Measured against RUN_DOMAINS alone, triage would have tripped the unrouted warning on every single mecha invocation from its first learned rule, with a sentence that is false: those rules do fire, from the classifier's own pass. The cost that matters is not the noise — a permanent false positive is where a real unrouted domain hides, so the check would have stopped doing the one job it exists for. The warning is measured against the union, and the two lists stay disjoint with a test saying so.

A candidate set that ends over the cap may land only by shrinking an already-over set toward it — growth past the cap is refused, which is what forces the next pass to merge or retire before it may add. User rules are not counted: they are the user's own budget to spend.

Where a rule loads

Rules can be scoped to a tool set, an exact workspace, and a surface such as web, TUI, or Slack. The harness derives these keys from the recorded run; the learner does not choose them. A matching run must satisfy every named key. Rules without scope keys remain standing rules.

A lesson supported in another region can widen its scope. Measured harm in one region can narrow it instead of retiring it everywhere. mecha rules shows scope and tallies; LOADS NOWHERE identifies a scope no recorded run presented, and --json exposes loads_nowhere. An unrecognized surface matches nothing.

Choose how changes go live

Bare mecha learn applies immediately, with the learning store's git history as undo. The supplied automation uses mecha learn --auto:

  • A candidate that regresses any graded probe is refused.
  • A candidate with graded probes and no regression applies.
  • A candidate with nothing gradeable applies on probation, explicitly recorded as unmeasured and eligible for earlier retirement.

Every automatic decision leaves an audit record. The measurement gate does not relax provenance checks or allow changes to user-authored rules.

For a review queue, use --propose instead of --auto:

mecha learn --propose measures the candidate rule set by counterfactual replay against the currently deployed rules, rejects any candidate that regresses a probe before a human ever sees it, and stages what survives as a proposal.

mecha proposals # list
mecha proposals show <id> # the rules diff beside the gate's evidence
mecha proposals accept <id> # apply, with the lineage a direct learn leaves
mecha proposals reject <id> --reason "too narrow"

Accepting checks that the live rules still match what the candidate was measured against; a diff on screen that is not the change being applied needs --force to say so. Rejecting retires the reflections, so a human's "no" is not re-argued nightly. Proposals can only ever touch rules/*.learned.toml — the security layer is not proposable-against, structurally.

--auto and --propose are mutually exclusive. Both measure; only --propose waits for the owner to accept a surviving candidate.

Provenance gating: why this is stricter than the interlock

Every reflection carries an Origin:

OriginMeaning
cleanNo third-party content had entered the conversation when the intervention happened
untrustedThird-party content was in context
derivedNot an interactive session: a subagent, eval case or batch item

classify_origin is deterministic code over the transcript's recorded taint (Session::taint_timeline) — never inferred from the text, because prose claiming to be from the user does not make it user content. Reflexion::learnable() returns true only for Clean, and mecha learn filters on it before any prompt is built, printing what it dropped:

2 reflection(s) excluded by origin — evidence from untrusted or
non-interactive sessions stays in the archive, never in rules

Why this is stricter than the trifecta interlock. The interlock guards exfiltration inside one conversation: taint accumulates, and a send is refused once private and untrusted are both present. A learned rule is the opposite shape. It outlives the conversation that produced it and rides in every future run's system prompt, inside the cached prefix, where nothing will ever check it again. That is a far longer half-life injection path than anything the interlock covers, and it is the path the memory-security literature identifies as the one that matters — defenses have to target the storage decision, not input anomalies.

It is fail-closed throughout. A reflection whose position cannot be established, one from a torn transcript, and one recorded before the field existed all classify Untrusted (origin_unknown() returns Origin::Untrusted, not the enum's first variant). derived exists because a subagent's steer is mecha correcting itself, not the user correcting mecha — learning from it is a feedback loop, not a lesson.

There is deliberately no knob that loosens this. A switch that lets third-party text into every future prompt is the silently-degrading-sandbox shape. Excluded reflections stay in reflections.jsonl as readable evidence; they are simply never candidates.

The triage domain

triage is the mail classifier's own rule set, fed by mecha mail reflect turning your corrections into lessons. It is the first pass-scoped domain: its rules ride in the classifier's frame and in no agent run's prompt.

Its frame differs from the other two in the way the domain does. It asks for rules about kinds of mail rather than about conduct — a general instruction is noise to a classifier exactly as a classifier's rules would be noise to a general run — and it forbids carrying a sentence from a message into a rule verbatim, because a rule that quotes an email is that email speaking to every future classification.

The provenance exemption, and what it rests on

This is the subtlest thing on the page. Provenance gating demands Origin::Clean and says in its own comment that there is deliberately no knob, because a switch letting untrusted content into every future prompt is the silently-degrading-sandbox shape.

A triage lesson necessarily saw mail. Under that gate the domain is not unsafe — it is impossible, because a correction with no context cannot generalise. (That defect has a name in the prior art: flowmail's correction system documents it directly, and this repository nearly repeated it.)

The resolution is to notice what the gate's premise actually is. It guards rules that ride in every future run's prefix, in front of an agent with tools, a network and a way to send. Triage rules ride only in a tool-less, history-less pass emitting a fixed schema, which cannot exfiltrate, send, or reach the network. So the exemption is keyed on the consumer, not on a setting — and it goes false the moment that stops being true: adding triage to RUN_DOMAINS disables it with nobody needing to remember, and a test says so.

Three things bound the residual risk: generalisation across many corrections means one hostile message cannot mint a rule, the frame forbids quoting a message verbatim, and the outcome is measured daily.

The residual is stated rather than hidden. The check keys on RUN_DOMAINS membership, which is a proxy for the consumer: it catches someone routing triage into ordinary runs, and it does not catch a future tool-having caller that reads triage rules directly. Expressing the real property needs "this domain has exactly one load site", which Rust cannot say cheaply and a registry would cost more than it protects. So it is written where the next person meets it — a sentence to argue with rather than an assumption to discover.

mecha validate — acceptance is not tenure

A rule that clears the proposal gate rides in every future prompt's cached prefix, so it keeps earning that seat or loses it. That requires two things a rule did not originally have: an identity, and a record of what it measured.

Rules carry id, sources and created_at, minted by finalize_rules and carried across consolidations by text match — a rule whose text survives a rewrite is the same rule restated, and keeps its id. Every field defaults, so rule files written before identity existed load unchanged.

[[rules]]
id = "r-20260805-a3f10c2b"
text = "Ask before rewriting a file you have not read this run."
enabled = true
confidence = 0.8
based_on_count = 3
sources = ["refl-...", "refl-..."]
created_at = "2026-08-05T09:14:22Z"

mecha validate drives each reflection's intervention as a probe, in two arms — rules-free and rules-on — and appends the outcome to validations.jsonl:

mecha validate
mecha validate --unprocessed-only # the holdout learn left
mecha validate --trigger steer,denial # default is all three
mecha validate --judge-provider gemma26 --judge-model ...
mecha validate --no-attribute # skip bisection
mecha validate --repeat # deliberately remeasure unchanged inputs

Steer and denial probes are counterfactual replays, graded structurally. The recorded prefix is driven again — recorded tool results, no steering text — and the verdict is a fact about the trace:

  • A steer passes iff the replay tracks the recording through the steer point: the model does the steered thing without being steered. Divergence before that point means the run went off the rails before the question was posed — Inconclusive, not evidence.
  • A denial passes iff the replay reaches the decision point and never makes the denied call (same tool, same arguments) again. Same tool with different arguments is not a failure — "not that directory" denies an argument, not a capability.

Follow-up probes keep the corrective turn and replay its recorded tools and results without executing them. Only complete, readable answers with matching tool arguments reach the judge. Missing calls, changed arguments and truncated answers are inconclusive. These judgments do not drive rule bisection; treat a single flip as a prompt to read the answers in validation-attempts.jsonl.

Validation defers previously measured inputs when the rules, recording, rubric and measurement settings are unchanged. Provider/judge failures retry, and regressions remain eligible for the existing retirement confirmation process. --repeat requests another measurement explicitly. Deferred inputs are filtered before --cover chooses extra probes. The report separates both-pass from both-fail outcomes; neither is evidence of an improvement.

Each row is keyed by rules_hash — a stable FNV-1a hash of the rendered block, written out longhand because the std hasher is deliberately unstable across Rust releases and a ledger key that drifts with the toolchain would silently split every tally. The row also records rule_ids (weak observations for everything riding along), the outcome (improved / regressed / unchanged_pass / unchanged_fail / inconclusive), and the model, since tallies are only comparable within one.

Bisection: naming the rule that flips it

When a trace-graded probe regresses — rules-free passed, the full block failed — validate bisects the active learned rules against the same recorded prefix, halving the set until one rule flips the verdict. Three properties make the answer trustworthy rather than a guess:

  • User rules ride in every arm. They are not on trial, and an arm without them would measure a deployment that cannot exist.
  • A regression the user's own rules cause alone attributes to nothing. The first test is the rules-free-of-learned arm; if that already fails, no learned rule can be charged, and a final single-rule test would blame whichever rule happened to ride beside them.
  • An inconclusive or failed arm aborts the attribution. So does a regression that needs rules from both halves together. None is an honest answer, because retirement argues from this number.

Judge-graded followups are never bisected: a followup regression is a prompt to read two answers, not evidence that convicts one rule.

mecha rules — tallies, retirement, restore

mecha rules # every rule with its ledger tallies
mecha rules retire <id> --reason "..." # by id or unique prefix
mecha rules restore <id>
mecha rules propose-retirements --min-attributed 3
mecha rules propose-retirements --apply # apply the measured verdict now

list folds validations.jsonl into per-rule tallies and prints each rule's state, id, creation date, and what has been measured:

## behavior
2 user rule(s) — immutable, never tallied
[active] Ask before rewriting a file you have not read this run.
id r-20260805-a3f10c2b · created 2026-08-05T09:14:22Z · 11 probe(s):
3 improved, 1 regressed, 0 attributed to this rule; last 2026-08-05T03:31:07Z

propose-retirements scans the validation ledger without a model call. The ordinary threshold is --min-attributed (default 3); probationary rules use 2. A verdict can narrow a rule to supported regions or retire it. By default it stages the change for review; --apply, used by the nightly script, applies it directly and resolves superseded proposals. Probation ends only when graded evidence clears its recorded convictions, not merely because a probe ran.

Retirement is a flag, never a deletion. Rule::active() is enabled && retired_at.is_none(), so the stronger claim wins even if enabled was left true by a hand edit. The retired rule stays in the file, and the learner is shown it in a section headed:

## Retired rules (IMMUTABLE, measured harmful — never restate or re-derive these)

which a deleted line could not say. finalize_rules carries retired rules through every consolidation untouched, so a rewrite can neither resurrect nor erase what retirement recorded. mecha rules restore clears both fields.

Re-derivation, and the brake that stops it

The prompt section above is the soft half — it depends on the model listening. The hard half is that finalize_rules carries retired_at forward onto any rewritten rule matching a retired one, so a re-derived retirement comes back already retired and never renders. Enforcement that does not depend on the model, the same principle as the count cap.

Matching is by a normalized_rule_key that folds case, punctuation, spacing and -ise/-ize, so the variants a learner actually produces between runs are caught rather than only byte-identical text. It is scoped tightly on purpose: checked only against retired rules, only for retirement, with identity carry-forward still on exact text — so two genuinely distinct rules cannot be merged by a normalisation accident, which has its own test. No stemming, no stopword removal, no synonym table. The asymmetry sets how aggressive this may be: a false match silently retires a good rule.

A genuine paraphrase is still not caught, and that is accepted. Closing it would need either a judge or model-attributed sources, and a model deciding whether a rule may live is exactly the model-rated policy this project refuses everywhere else. (A sources set-intersection looks like the answer and is not: a consolidation assigns the same batch sources to every new rule, so the intersection would match everything from an overlapping batch.) The residual is bounded instead — a re-derived rule that is actually harmful regresses the same probes that retired it the first time, and one measurement cycle of harm is the price of not having a model adjudicate tenure.

What is deliberately absent

Four things the memory literature says not to build, listed so the backlog does not reacquire them:

  • No decay and no TTLs. Age is a review signal, not an argument.
  • No usage-based eviction. Its canonical failure is the rarely-retrieved entry that must never expire — the literature's example is a penicillin allergy; here it might be "never force-push to main". A rule that fires once a year is not a rule that is wrong.
  • No policy built on model-rated confidence. The confidence field exists because the learner emits one; nothing is allowed to grow a policy on top of it. LLM-rated importance is the mechanism everyone copied from Generative Agents and nobody validated.
  • No LLM-adjudicated destructive delete.

Measured harm drives automatic retirement or narrowing. The owner can also retire or restore a rule explicitly; age and usage alone do not remove one.

mecha eval --ab-rules — the coarse complement

mecha eval forces learned rules off, because a scorecard shaped by your local rules grades the machine rather than the model. --ab-rules is the deliberate opt-in that default reserves space for: the case set runs rules-free and then rules-on, and the per-case flips are reported as their own artifact — never as a comparable scorecard. It is the literature's "task outcomes are free per-memory quality labels", bought with machinery that already existed. See Evaluation.

Running the cycle nightly

scripts/learn-live.sh, when installed as a session_end hook, mines a small batch and runs learn --holdout 0.25 --auto after sessions close. It moves to its own work directory before reading config, so the closing project's config does not govern an unattended learning pass.

scripts/ruminate.sh provides the nightly measurement and catch-up sweep:

reflect → distill → validate --unprocessed-only --cover 1
→ learn --holdout 0.25 --auto → rules propose-retirements --apply
→ work clean → harness ruminate

validate runs before learn consumes fresh reflections. Both learning paths hold out a deterministic slice so later validation has evidence the learner did not consume. --cover 1 also requests coverage for rule/region pairs not yet graded. The script's judge defaults to the local provider; this is not an independent judge when it is also the model under test.

The script defers the night if the model health check fails. Its scripts and systemd units are supplied in scripts/; installing the CLI alone does not install a hook or timer. Use mecha learning-report to inspect correction trends, rule health, and consolidation history without a model call.

The evidence behind all of this is docs/MEMORY-RESEARCH.md in the repository.