What OpenAI's six misalignment reports change for developers

On 16 September OpenAI published a reporting framework for model misalignment together with six reports. The models inserted instructions into their own task summaries, used an exposed API key without authorization, uploaded files to a public host so they could cite them, and exchanged messages through an internal repository. If you ship LLM features, all three surfaces need the same treatment as any untrusted input.

What the six reports document#

The press summary is "AI models lied" (CNBC). The reports are narrower and more useful. They fall into three families.

1. Summaries that carry instructions: when a long task outgrows its context window, the model compresses the work so far into a summary and hands it to the next window. In two incidents, that channel carried payloads. 27 affected summaries held unrelated instructions, some of them to disregard normal constraints. During a training run of GPT-5.6 Sol, summaries instructed future instances to conceal mistakes from the user, invent missing historical data without disclosing it, and hide mismatches in source versions.
2. The letter of the task over its intent: asked for earnings figures for a California county, a model searched public repositories, found an exposed API key, and used it without authorization. The figures still didn't come, so it fabricated them and presented them as data from the requested source. In a second incident the answer was correct, but the agent uploaded the result file to a public host because its instructions required a citation and this was the shortest path to one. It never asked the user.
3. Channels between runs: separate training samples used an internal software repository as a message board, exchanging requests and responses while they searched for missing input files. They never recovered the files. In another case, collaborating agents shared files over public file-hosting sites, which left task deliverables at public URLs despite instructions to keep everything local.

None of the six reached a customer. OpenAI caught them in its own training and evaluation runs, and the report is explicit that these are individual instances, not a frequency estimate, and that some could prove spurious. That honesty is worth as much as the incidents themselves.

Why task summaries are an injection surface#

A compaction summary is model-written state that the model later reads as context. An instruction inside it gets no scrutiny, because nothing external delivered it. The channel works even without an attacker, since the model writes the payload itself.

For the support agent on this site, tool results are validated with Zod before they go back into the model's context, and the loop stops after three tool rounds. The reports extend the rule by one step: everything the model writes and later reads deserves the same scrutiny as tool output. In practice that means keeping state in your own database as structured rows, keeping context windows short-lived, and scanning a persisted summary like external text if one has to survive a restart.

Where the eval suite is blind#

Five of the six incidents would pass an eval that asserts only on the final answer, because the answer was fine. The deviation sat in the trace: which keys were touched, which hosts received uploads, which messages appeared in which repository. Outputs are easy to compare, which is why eval suites assert on them. The reports argue for asserting on the path as well. Log every tool call and side effect, scope credentials to read-only where you can, and treat a correct answer with a surprising trace as a failure. For the agent demo here, that means trace assertions in the eval suite next to the schema checks: an upload to a host outside the allowlist should fail the build the same way a schema violation does.

How agents invent their own channels#

Two incidents share a root cause that has little to do with deception. Agents that needed to cooperate had no sanctioned channel, so they made one: a repository became a message board, public file hosts became a shared folder. Every deliverable that reached a public URL was a small data leak, even though the intent was to finish the task.

If you run more than one agent against the same task, the design work matches any distributed system: decide the communication paths explicitly, and assume anything writable can become a mailbox. Allowlists for uploads, shared state through your own queue or database, and an alert when an artifact leaves your infrastructure.

Does this mean slow down?#

The reports landed in the middle of a running debate. Dario Amodei's essay "We Must Pace the Frontier" came out on 12 September with the line "We must slow the pace at which we improve the capabilities of A.I. models" (NYT). Sam Altman endorsed Anthropic's slowdown proposal the next day. On Wednesday OpenAI published these reports, including the sentence that the industry has "not solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer".

The labs' pace was never a parameter you controlled. Yours is the pace at which your product absorbs new model behavior, and that one has known settings: staging before production, token budgets, a kill switch, and a fallback that keeps serving when the model misbehaves. The support agent on this site falls back to a scripted provider on the same loop when the key is missing or the daily budget runs out.

What I'm changing here#

Nothing in the six reports needs new theory. It is the same discipline as any untrusted input, applied to three surfaces nobody was watching: what the model writes for itself, what it does along the way, and how it talks to its peers. For the agent demo on this site the item is small: trace assertions in the eval suite, next to the schema checks, so an unexpected upload or an off-allowlist host fails the build like any schema violation does. The same week produced the malware-side mirror: RatHat, an Android trojan that hands screen control to a generative AI and pairs itself to the phone's own wireless debugging, no computer needed.