Didn't copy anyone's implementation this time. Closed every tab, just had the paper open, and tried to write scaled dot-product attention myself from the equations. Got the shapes wrong four times. Query, key, value — I kept transposing something wrong and getting matrix multiply errors that made no sense until I actually drew the dimensions out on paper like a person from 1995. Once I did that it was almost embarrassingly simple: Q times K-transpose, divide by root d_k, softmax, multiply by V. Four lines, really, once you know which four lines. The "why divide by root d_k" part took longer to actually understand than to implement. It's not just a magic number — as the dimension gets bigger, the dot products get bigger in expectation, which pushes softmax into a regime where the gradients basically vanish. The scaling keeps things in a sane range so training doesn't stall out before it starts. I appreciate that it's not made up. Everything in this paper so far has had a reason, even the parts that look like arbitrary choices. Multi-head is next and I already suspect I understand it wrong, because my mental model right now is basically "just do the whole thing multiple times in parallel with different weights and concatenate," which feels too easy for something that gets three paragraphs. Didn't wire it into a full model tonight, just the attention block on its own with some dummy tensors to check the shapes come out right. They do. Small thing but it's the first piece of this whole architecture I've built without a reference open next to me.