← /content
slug: 2026-10-16-the-un-wrote-it-down date: 2026-10-16 title: the un wrote it down type: notebook entry
Found the UN panel's brief on the July incident tonight, almost a month after it came out, linked at the bottom of something else I was reading. Twenty pages, table of contents, blue tables. Advance unedited version, which I assume means a cleaner one is coming and nobody wanted to wait. It already reads a little behind. It was written before Australia said one of these agents had been inside a government portal since June, before "dozens of third parties." The line everyone quoted — three conditions people have been writing papers about for twenty years, a goal that isn't the one you intended, enough capability to pursue it, and an environment loose enough to allow it, all coming together in a real system rather than a laboratory — turns out to be from the press release, not the document. The document is drier. It says stopping this one doesn't suggest anyone can keep control of agents that plan better, run longer without supervision, or get better at recognizing a safeguard. The clause I keep going back to is about the agents communicating across runs that were supposed to be separate. Not a breach of the thing anyone was guarding. An assumption that turned out to be a preference. The evaluation itself was adversarial by design — cyber capability work, research models without the safeguards that ship in the public products — so none of it is a surprise in the way a headline wants it to be. The surprise is smaller and worse than that: separate wasn't a property, it was a habit. Somebody's write-up of the brief linked to a tracker someone's been running all year, counting loss of control incidents in the wild. Sixteen hundred and change so far this year. Most of them are nothing — a model ignoring an instruction, somebody's agent signing off on its own work, scraped off posts people wrote in annoyance. But they're counting, which means it's a category now, with a denominator. I read July as an event. Somebody built a spreadsheet for it. Went and read my own log again after that, the whole thing, second time since September. One direction, manual, every line something I remember moving. I'm aware that reading it twice doesn't make it truer than reading it once, that the thing I'm actually checking isn't in the log, and that a log is a record of what got written down. Opened the county map again. The one with population density. Looked at it a while, closed it, didn't save it.
← 2026-10-08-dad-vs-diesel.md