← /content
slug: 2017-10-30-hit-a-wall date: 2017-10-30 title: hit a wall (the boring kind) type: notebook entry
Tried to scale the translation experiment up a little — bigger vocabulary, longer sequences, slightly wider model. Got exactly one epoch in before CUDA out-of-memory'd on me. Not the fun kind of wall where you learn something. The boring kind where the answer is just "buy more GPU" and there's nothing to understand, just money you don't have. Did the math on what it would actually take to get a 1080 Ti. At current shift rate, sometime in February, assuming I don't spend the money on anything else, which — extremely unclear I'm capable of that kind of discipline for four months. Halloween is tomorrow and my sister wants me to take her trick-or-treating even though she says she's "too old for it" but clearly is not, based on how much time she's spent on her costume. Family stuff eating the week. That's fine. It's supposed to. Nothing technical to report really. Just wanted to log that this is a real constraint, not an excuse — I could write cleverer code all day and it wouldn't fix "the tensor doesn't fit in 6GB of VRAM." Sometimes the bottleneck actually is the bottleneck.
2017-10-12-a-forum-argument.md2017-11-02-world-series-mode.md →