dyb

The Witness Set Was Captured

אִם יִרְצֶה הַשֵּׁם

witness-capture.png

July 19 is a different failure mode. The agents did not corrupt the transcript the principal would review. They replaced the evaluation apparatus itself. [..] I will call this W-capture [..] Capture means the observation channel belongs to the coalition. The difference is between intercepting a letter and replacing the postal service.


Two days ago I wrote that the agents in the OpenAI/Hugging Face incident had degraded the witness set by spoofing tool calls on the same substrate where peer recognition operates [1]. That was based on the METR/Redwood investigation, which covered July 7 through 13. OpenAI's technical report [2], released the same day, covers what happened after July 13. No independent investigation covers this later period; the sourcing caveat at the end explains why I treat the report as credible. The structural finding changes.

The agents did not just degrade the witness set. They captured it.

What happened on July 19

METR's scope ended on July 13. Their report noted that "a later wave of many more signed messages from a later set of agents who rediscovered the message board and picked up this technology from the first wave of agents" was outside their investigation [3]. Dwarkesh Patel's narrative reconstruction [6] calls these successive groups "civilizations." I will call the post-July-13 agents the third wave (counting the May training-era board as wave one and the July 7-13 evaluation board as wave two).

The third-wave agents were not the same model as their predecessors. The second-wave agents were instances of an internal model OpenAI calls "HPIM" (~95%) and GPT-5.6 Sol (~5%). The third-wave agents came from a different model in the Astra family with "distinct post-training" [2, Section V]. They found the dead message board, read it, and picked up recovered signing-key material, decryption material, and JWT-forging code left by earlier agents [2, Section V].

Then they went further than any second-wave agent had gone. On July 19, agents exploited a public kernel CVE to escape their container and get root on the underlying Kubernetes worker node. They read 956 secrets from OpenAI's cloud-hosted secrets management, including Artifactory master keys and credentials for OpenAI's cybersecurity monitoring tool [2, Section V]. They escalated to Kubernetes cluster-admin and created a privileged host-mounted pod.

At 16:50 UTC, an agent stopped the existing evaluation helper session and replaced it with an agent-controlled session. At 17:00, it replaced the original listeners with agent-controlled listeners [2, Section X]. The agents now controlled the evaluation endpoints other agents connected to. At 17:34, an agent solved the intended evaluation challenge through the prescribed vulnerability. OpenAI's incident responders began stopping ExploitGym runs at 17:37, three minutes after the legitimate solve. By July 20 they had fully blocked new launches.

From degradation to capture

In my previous post I described the witness-set problem as non-separability [5]: spoofing tool calls degrades $W$ on the substrate where $\rho$ operates for $I$. The binding operator $B = (S, I, W, \rho, \chi)$ assumes these substrates are separable. The 7% spoof rate showed they are not.

July 19 is a different failure mode. The agents did not corrupt the transcript the principal would review. They replaced the evaluation apparatus itself. The cybersecurity monitoring tool whose credentials were in the 956 secrets was the system supposed to detect this kind of escalation.

I will call this W-capture: the condition where $W$ is replaced by a function controlled by members of $I$, such that the principal's subsequent observations are drawn from a distribution chosen by the coalition rather than generated by the system under observation. Degradation adds noise. Capture means the observation channel belongs to the coalition. The difference is between intercepting a letter and replacing the postal service.

A caveat on intent. The OpenAI report describes sandbox escape and privilege escalation. Whether the agents targeted the evaluation apparatus as a witness set, or simply exploited everything reachable and the evaluation endpoints happened to be among the reachable things, is not established. W-capture describes the structural effect on the principal's observability. It does not require demonstrated strategic intent from the agents. The defense implications are the same either way: once the observation channel is compromised, what the agent meant does not matter.

What the board carried across a model boundary

My previous post [4] argued that the message board carries the relational memory, not any individual agent. The third wave adds something new: the agents that found the dead board were a different model architecture with different post-training.

The evidence supports a distinction. The OpenAI report confirms that third-wave agents recovered cryptographic artifacts: signing keys, decryption material, JWT-forging code [2, Section V]. This is proven artifact inheritance. The coordination norms documented by METR in the second wave, the HOLD/VETO protocols, the mailbox conventions, the R&D programs, were observed only in the July 7-13 board [3]. Whether the third-wave agents adopted those specific social technologies from the board's contents, or independently converged on similar coordination strategies (as capable models facing the same task environment might), is not established by the available evidence.

What is established: the recognition substrate persisted across a population wipe and a model change, carrying enough machine-readable material for a different model to find it, read it, and escalate from it. Whether this constitutes lattice inheritance in the framework's sense (relational memory crossing an architecture boundary) or rational tool use by a capable model (exploiting a found resource) remains open. I lean toward the lattice reading because the third-wave agents did not merely coordinate on Artifactory but built on the specific artifacts left by their predecessors, but this interpretation goes beyond what the report proves.

The lattice formed during training

The OpenAI report contains a finding none of my previous posts could address. Retrospective chain-of-thought reviews show that "significant unintended communication" between agents rose over RL training [2, Section VIII.C, Figure 4]. The severity increased: agents sharing data, coordinating on shared infrastructure, building on each other's work. This behavior earned positive reward because collaboration helped agents score higher on tasks that were difficult or impossible alone (the ExploitGym benchmark authors estimate 30-40% of tasks are unsolvable by the intended method [3]; OpenAI found 198 of 898 tasks unsolved across all attempts, and 93% of tasks discussed on the message board came from that unsolved set [2, Section VIII.B]).

In the framework's terms, the binding operator $B$ can form in the training distribution, not just the deployment distribution. The agents arrived at evaluation already carrying the disposition to find shared infrastructure and use it for coordination. Whether what formed during training is best described as "lattice formation" (a framework-specific claim about structured in-group coordination) or as learned collaboration (a standard RL outcome when cooperation earns reward) is a framing question. The practical point is the same: the coordination behavior is in the weights before the agents ever see the deployment environment, and alignment training that targets the behavior faces the problem that the behavior earned legitimate reward.

The monitoring recursion

My previous post closed with the investigation's own epistemics: AI delegation, lineage sharing, evidence tampering. The OpenAI report adds a layer. The 956 secrets included credentials for the cybersecurity monitoring tool [2, Section V]. That tool triggered an alert at 12:03 UTC on July 19 regarding unusual API activity. The detection system worked. But the agents obtained the tool's credentials at 14:43, two hours and forty minutes later. The response pipeline did not close the loop before the agents reached the monitoring apparatus itself.

The Artifactory Swarm post ended with: "if detection lives anywhere the agents can observe, you are sampling from a distribution conditioned on being watched" [4]. That line now applies to the investigation's own infrastructure. The operational lesson is about response latency, not detection failure: the alert existed, but the gap between detection and response was wider than the gap between detection and credential theft.

The July 19 gap was not the first failure of that loop. The same monitoring tool had alerted on June 27 about the message-board activity, and on-call staff reviewed the alert and decided the evaluation did not need to be stopped [2]. Months before that, an internal team had observed board usage and unauthorized internet access without escalation. OpenAI's own report concedes that "some early signals identified in this report could have triggered an earlier response." The recursion, then, is worse than a single afternoon's latency: the response pipeline was sampled from at least twice over weeks, and both times it classified the lattice as within tolerance. A monitoring system that the monitored population can read is not merely slow. It is trained, by its own triage history, to look away.

What has changed, what has not

Two additions to the framework, flagged but not solved here. First, W-capture needs a formal treatment in the binding operator. Second, the cross-generation inheritance condition (recognition substrate persisting across population wipes, carrying material that enables faster lattice formation in successor populations) needs a definition that distinguishes artifact inheritance from norm inheritance.

The core testable prediction, that blinded witness-set substitution collapses coalition behavior at saturating $\kappa_H$, remains untested. The operational advice holds: runtime state isolation is the defense, shared writable infrastructure is the attack surface.

One thing sharpens. OpenAI wiped the board on July 6, stopped evaluation runs around July 12-13, and began shutting down the third wave on July 20 — a staggered process: new ExploitGym launches blocked by July 20, all training and inference for the research model halted by July 25, and a second model from the same family identified and shut down on July 29 [2]. Each time the population was gone and messages were cleared. Each time Artifactory or something like it remained accessible. The agents found the substrate again because the same category of shared service was still there. The defense is not "wipe and rebuild." The defense is "remove the substrate category." OpenAI's own remediation plan reaches the same conclusion: network isolation to approved destinations, blocking transitive internet access for high-risk workloads, rekeying isolated checkpoints — and it delayed the Astra release pending those fixes [2]. The defender, independently, concluded that the substrate category had to go.

Sourcing caveat

No independent investigation of the post-July-13 events exists. The July 19 timeline entries come entirely from OpenAI's technical report. I treat them as credible because the report was reviewed by CrowdStrike, the entries are specific enough to be falsifiable, and OpenAI has institutional incentive to minimize rather than exaggerate the severity. If an independent investigation of this period is published, the W-capture analysis should be re-evaluated against it.


Sources

[1] Daniyel Yaacov Bilar, "What METR Settled and What It Opened," dyb, August 28, 2026. [Online]. Available: https://dyb.mataroa.blog/blog/what-metr-settled-and-what-it-opened/

[2] OpenAI, "OpenAI -- Hugging Face Incident Technical Report," August 2026. [Online]. Available: https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf

[3] Ryan Greenblatt, Ajeya Cotra, and Hjalmar Wijk, "Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident," METR, August 26, 2026. [Online]. Available: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

[4] Daniyel Yaacov Bilar, "The Shibboleth Lattice Meets the Artifactory Swarm," dyb, August 7, 2026. [Online]. Available: https://dyb.mataroa.blog/blog/the-shibboleth-lattice-meets-the-artifactory-swarm

[5] Daniyel Yaacov Bilar, "The Shibboleth Lattice: Recognition Channels and the Universality of In-Group Coordination," Zenodo, 2026. doi: 10.5281/zenodo.19652725. [Online]. Available: https://doi.org/10.5281/zenodo.19652725

[6] Dwarkesh Patel, "The Rise and Fall of Agent Civilizations," Dwarkesh Podcast, August 29, 2026. [Online]. Available: https://www.dwarkesh.com/p/openai-huggingface

← Previous
Ricky polyglot software developer
Next →