Small thing today. Got the toy repo actually running end to end — forward pass, loss computing, no crashes. Doesn't mean anything yet, loss is basically random, but the pipe is not broken anymore, which after last week feels like an achievement. Tried to understand multi-head attention properly instead of just accepting "more heads = more good." Best explanation I've found so far is that each head gets to learn its own subspace to compare things in — like giving the model several different lenses to look at the same sentence through instead of one. One head might end up tracking something like subject-verb relationships, another might track nearby words, who knows what else. Nobody tells it what to specialize in, it just ends up specializing because different lenses end up useful for different things during training. I don't fully buy my own explanation yet. Going to try visualizing what different heads actually attend to once I have something trained on real data instead of random noise. If they don't specialize into anything interpretable I'll know I'm wrong about this. Bought a $9 fan from the hardware store (employee discount doesn't kick in til I'm actually hired, apparently, still just "helping out unofficially" per Dave) to point at the GPU. Room gets genuinely hot when it's training for more than an hour. July was a bad month to start this hobby, temperature-wise.