wasshuber

wasshuber

Programming Machine Learning: MNIST benchmark for multi-layer networks

Does anybody have benchmark results on what accuracy is achievable on the MNIST data with a multilayer network? I am particularly interested in smaller node numbers but deeper networks. For example what can be done with two, three or four layers of 100 nodes? Or similar.

I have extended the one hidden layer code to multiple hidden layers. Now I am wondering how much better the results should get. For example, with a 100-node hidden layer the book and my own experimentations achieve 98.6%. How much better should it get if one adds a second 100-node layer? I am asking because my early results do not show much if any improvement. I have even upgraded the SGD to the Adam algorithm which is a lot faster but the final accuracy it achieves is pretty much identical, perhaps slightly higher by 0.05% or so.

Most Liked

wasshuber

wasshuber

Thanks. Makes sense. So for dropout one essentially pretends that node doesn’t exist.

It is true, it gets a bit harder for some algorithms, but then again I think it is just one more tensor multiplication added. However, other advanced algorithms are quite easy to implement. For example, I implemented the Adam optimizer. This was straightforward and just a few lines of code, and it provides a wonderful speed-up.

Where Next?

Popular Pragmatic Bookshelf topics Top

jamis
The following is cross-posted from the original Ray Tracer Challenge forum, from a post by garfieldnate. I’m cross-posting it so that the...
New
simonpeter
When I try the command to create a pair of migration files I get an error. user=> (create-migration "guestbook") Execution error (Ill...
New
raul
Page 28: It implements io.ReaderAt on the store type. Sorry if it’s a dumb question but was the io.ReaderAt supposed to be io.ReadAt? ...
New
rmurray10127
Title: Intuitive Python: docker run… denied error (page 2) Attempted to run the docker command in both CLI and Powershell PS C:\Users\r...
New
New
swlaschin
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
New
jskubick
I’m running Android Studio “Arctic Fox” 2020.3.1 Patch 2, and I’m embarrassed to admit that I only made it to page 8 before running into ...
New
akraut
The markup used to display the uploaded image results in a Phoenix.LiveView.HTMLTokenizer.ParseError error. lib/pento_web/live/product_l...
New
taguniversalmachine
Hi, I am getting an error I cannot figure out on my test. I have what I think is the exact code from the book, other than I changed “us...
New
Henrai
Hi, I’m working on the Chapter 8 of the book. After I add add the point_offset, I’m still able to see acne: In the image above, I re...
New

Other popular topics Top

axelson
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
New
AstonJ
SpaceVim seems to be gaining in features and popularity and I just wondered how it compares with SpaceMacs in 2020 - anyone have any thou...
New
AstonJ
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
New
AstonJ
If you are experiencing Rails console using 100% CPU on your dev machine, then updating your development and test gems might fix the issu...
New
AstonJ
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
New
Maartz
Hi folks, I don’t know if I saw this here but, here’s a new programming language, called Roc Reminds me a bit of Elm and thus Haskell. ...
New
foxtrottwist
A few weeks ago I started using Warp a terminal written in rust. Though in it’s current state of development there are a few caveats (tab...
New
PragmaticBookshelf
Build efficient applications that exploit the unique benefits of a pure functional language, learning from an engineer who uses Haskell t...
New
PragmaticBookshelf
Author Spotlight: Peter Ullrich @PJUllrich Data is at the core of every business, but it is useless if nobody can access and analyze ...
New
New

Sub Categories: