It sounds stupid, but I can honestly say that figuring out how neural nets work and making one of my own has kind of been a life-goal, and now I've finally done it. I have a toy network that achieves 95% classification accuracy (given relatively few epochs) on a very simple non-linear rule. Yay! Now I finally feel like I can talk about neural networks and training regimes like I know what I'm talking about! =)
For the longest, longest time -- like months -- it was working, but not well: it was achieving ~75% accuracy on the same simple task. When I tried common NN libraries under similar circumstances, they were hitting 95%, which seemed more reasonable. I finally figured out what the problem was: I wasn't using a bias unit, and apparently those libraries were -- I assume anyway, because now that I've added a bias unit, my accuracy jumped to a comparable 95%, which is what I thought it should be.
I would post it, but it's not good code. I'm just thrilled it works.
