Does anybody have benchmark results on what accuracy is achievable on the MNIST data with a multilayer network? I am particularly interested in smaller node numbers but deeper networks. For example what can be done with two, three or four layers of 100 nodes? Or similar.
I have extended the one hidden layer code to multiple hidden layers. Now I am wondering how much better the results should get. For example, with a 100-node hidden layer the book and my own experimentations achieve 98.6%. How much better should it get if one adds a second 100-node layer? I am asking because my early results do not show much if any improvement. I have even upgraded the SGD to the Adam algorithm which is a lot faster but the final accuracy it achieves is pretty much identical, perhaps slightly higher by 0.05% or so.