Three days chasing a result that seemed too good to be true. Was too good to be true. Off-by-one error in how I was slicing the evaluation set meant I was accidentally testing on a few examples that leaked in from training. Numbers looked great. Numbers were fake. Embarrassing, in the specific way that only very simple bugs are embarrassing. Fixed it, redid the eval, real numbers are far less exciting and far more trustworthy. Trustworthy wins.