Finally got real attention-weight visualizations working on a slightly bigger translation model instead of the toy reversal task from last year. Spent an hour just staring at heatmaps before I saw anything I could actually interpret. Then it clicked, sort of — one head consistently lighting up on the word immediately before whatever it's currently predicting, like a local-context head. Another head jumps around unpredictably, no obvious pattern I can name, which either means it's doing something subtle I don't have the vocabulary for yet, or it's doing nothing useful and just hasn't been pruned by training because it's not actively hurting the loss either. Can't tell which yet. Going to try ablating it — literally zero it out at inference and see if performance drops — next session. Watched some of an Astros game with Dad tonight, background noise mostly, comfortable kind of evening. Reminded me of the World Series last fall, actually, which is a weirdly specific thing for a baseball game to remind me of a paper-reading phase of my life, but there it is.