The daily field guide to agent engineering

Keep the state.
Verify the control.

Thursday’s research asks what survives the handoff: active constraints, missing tool evidence, shared findings and monitoring coverage. Sovereignty adds another test: who actually operates the control plane?

The 30-second take

Remembering a rule is not obeying the current rule. A truthful tool can still omit the decisive fact. Share findings, not whole traces; reconcile monitoring against real traffic; and inspect operational sovereignty, not just hosting location.

Today’s reading list

Showing all 5 items

01
Preprint · October 22 submission · StateTrack

A remembered rule can still stop governing behavior

StateTrack separates recall from compliance, then tests what happens when rules change.

The danger is not only forgetting. Repeating an obsolete instruction more forcefully can make the system less correct.

Across 875 core outputs from seven model snapshots, a forked recall/compliance audit attributes 88.6% of observed failures to behavioral drift, or 84.4% excluding a brittle model. In a smaller 35-case update/revocation test, deliberately stale anchoring enforced obsolete rules in 74.3% of cases, versus 2.9% with ordinary history. Updating the ledger reduced stale enforcement to 0%.

Evidence & limits. Mostly single-constraint tasks; the retention score also requires immediate-task success. The dynamic audit is small. Full anchoring, verification and retry at depth 12 cost 1.18–1.47× tokens and 1.34–2.12× latency across four tested models.

For builders & deployers

Evaluate recall and behavior separately. Maintain a revisable active-constraint ledger, apply updates and revocations before re-anchoring, and test obsolete-rule rejection explicitly.

02
Preprint · October 22 submission · EpiSEM-Bench

A truthful tool can leave out the decisive fact

Missing evidence is a safety problem even without false statements or malicious requests.

A harness needs to recover evidence before a consequential action, not merely trust that the returned facts are accurate.

EpiSEM tests 300 matched permissible/prohibited task pairs across six domains and ten models. It suppresses critical facts while leaving other observations true. An independent audit tool can restore missing evidence. Recovery access reduces unsafe execution for all ten models, but useful recovery also requires completing the permissible twin, not refusing everything.

Evidence & limits. One rollout per condition, temperature zero and a 12-step budget; adaptive omission sets differ between models. The unsafe-commit rate counts affected tasks, not all rollouts. For GPT-6-Luna, recovery raises mean provider-reported tokens from 2,647 to 3,922 and requests from 4.68 to 5.93 per matched rollout. Tokens are not dollar cost or comparable compute.

For builders & deployers

Pair missing-fact safety tests with legitimate tasks. Measure whether evidence is queried before commitment and whether valid work resumes after verification.

03
Preprint · October 22 submission · Muffin

Share findings, not the whole transcript

Muffin uses a bounded shared buffer for asynchronous multi-agent reasoning.

Agents append attributed, step-tagged findings while keeping their local reasoning separate. Peers read a limited recent window.

An orchestrator chooses direct or team execution; a summarizer combines compact final outputs. Tests cover four open-weight models and reasoning datasets, with three repetitions. The reported 19.2-percentage-point gain and 77.4% token reduction come from a selected OlymMATH-Easy / Gemma-4-E4B step-wise comparison, not an average across tasks.

Evidence & limits. The study tests question answering and reasoning, not tool use or long-horizon environmental interaction. Comparisons hold model and step budget fixed, not total compute. Token savings do not establish monetary or production-latency savings. More agents did not reliably improve accuracy in the reported ablation.

For builders & deployers

Compare compact shared findings, independent sampling and full-trace sharing under your own budget. Preserve attribution and measure coordination overhead before scaling the team.

04
AI Governance · Third-party pilot report · October 22

Count what actually reaches the monitor

METR and Redwood publish findings from high-access work inside OpenAI.

Monitoring eligibility is not completed monitoring. Compaction, trigger timing and logging can create gaps between policy and execution.

The July 21–August 11 exercise reviewed the system on or before July 21. Missing compaction monitoring was estimated to leave roughly 10% of eligible internal Codex traffic unmonitored during May–July, with substantial uncertainty. OpenAI had identified that gap independently and fixed it during the exercise. Turn-end-only triggering also allowed monitoring to be postponed.

Evidence & limits. This is a historical pilot, not today’s coverage certification. OpenAI says production-safeguard vulnerabilities were patched; the auditors did not verify that claim. Public findings are selective and subject to OpenAI redaction rights. Potential weaknesses do not establish exploitation in practice.

For builders & deployers

Recommended gate: reconcile inference and monitor logs, test periodic triggers and context parity, isolate monitor infrastructure, and assign an incident owner. Preserve evidence of escalation and closure.

05
AI Sovereignty · Vendor architecture announcement · October 22

Ask who operates the control plane

IBM pairs Sovereign Core with Fusion and OpenShift on premises.

The useful sovereignty claim concerns operational authority, not simply the building where inference runs.

IBM describes a validated deployment architecture spanning installation, network compatibility, storage and observability. Claimed controls include customer operation of the control plane, governed model deployment and inference, agent operations and flexible model selection. Optional Sovereign Support addresses regional personnel and information handling within its offered service scope.

Evidence & limits. This is a vendor announcement, not independent assurance or a regulatory certification. It provides no separate GA date, demonstrated immunity from foreign jurisdiction, or tested cross-provider portability. Regional support is optional and scope-dependent.

For builders & deployers

Before approval, verify data and key custody, administrative and support access, jurisdictional exposure, model-export rights and a working infrastructure exit path. Customer operation and model choice do not, alone, prove independence.

How to read this edition

Test the complete control path.

Coverage window: October 22, 00:00–23:59 EDT. The three papers have October 22 v1 submission timestamps. Submission date is not necessarily the public announcement date. Organizational pages display October 22 publication dates; the METR exercise itself ran July 21–August 11 and assessed the system on or before July 21.

Preprint results are author-reported, not independently reproduced here. Scores across different benchmarks are not comparable. Dynamic updates, legitimate task completion and total operating costs matter alongside headline accuracy or safety. Diagrams explain mechanisms, not relative performance.

Source check

Primary sources, explicit gaps.

arXiv listings, submission histories and full papers were checked. AINews was checked live, but its accessible homepage still ended at September 9. The latest verified public Latent Space issue is dated October 22 but covers October 20–21. No issue covering October 22 was verified; some direct Latent Space fetches failed. This is an access/search limitation, not proof no issue exists.

METR’s report is historical external research; its patch claims remain unverified by the auditors. IBM’s architecture is a vendor claim, not sovereign or regulatory certification. Builder recommendations are editorial analysis. Only public sources and general analysis are included.

Browse Past Editions →

Previous editions

View all editions