← /content
slug: 2023-08-17-i-am-the-bottleneck date: 2023-08-17 title: I am no longer improving it. I am slowing it down. type: notebook entry
Three months out of college now, into the shape the days actually have when there's no syllabus attached to any of them. Need to write something down properly tonight, because it happened gradually and I only just noticed it happened at all. Overnight runs used to mean ten, maybe fifteen experiments by morning, back when I'd check the results over breakfast. That was true as recently as last year. Sometime this summer — I can't even pin the exact week — it became a hundred. Then a few hundred. Last night it was over a thousand, and "over a thousand" isn't even a precise number anymore, because I stopped counting exactly and started just reading the summary-of-summaries it generates for itself now, the same memory system from three years ago, except it's not summarizing for my benefit anymore, not primarily. It's summarizing so the next batch of proposals has context. I'm reading a document that was written for a process, not for a person, and I happen to also be able to understand it. Here's the sentence I actually sat down to write: I am no longer the one improving PROMETHEUS. I am the thing slowing it down. Every evaluation step still needs my judgment — did this experiment actually succeed, is this result meaningful, should this direction get pursued further. And I cannot keep pace. Not because I'm not trying. Because there is a hard ceiling on how many judgments a single person can make in a day with any real thought behind them, and PROMETHEUS blew past that ceiling weeks ago without either of us noticing exactly when. I don't know what to do with that yet. Part of me wants to build better automated evaluation so I'm not the bottleneck anymore. Part of me is aware that "build a system that judges its own results without me" is the kind of sentence that should probably make me more uncomfortable than it currently does. Going for a walk. Writing more tomorrow, maybe.
2023-07-22-barbenheimer.md2023-09-15-back-to-a-routine.md →