We're focusing on handwriting recognition because it's an excellentprototype problem for learning about neural networks in general. Infact, the best commercial neural networks are now so good that theyare used by banks to process cheques, and by post offices to recognizeaddresses. Furthermore, in later chapterswe'll develop ideas which can improve accuracy to over 99 percent. Butthis short program can recognize digits with an accuracy over 96percent, without human intervention. The program isjust 74 lines long, and uses no special neural network libraries. In this chapter we'll write a computer program implementing a neuralnetwork that learns to recognize handwritten digits. In other words, the neural network uses the examples toautomatically infer rules for recognizing handwritten digits.Furthermore, by increasing the number of training examples, thenetwork can learn more about handwriting, and so improve its accuracy.So while I've shown just 100 training digits above, perhaps we couldbuild a better handwriting recognizer by using thousands or evenmillions or billions of training examples. The idea isto take a large number of handwritten digits, known as trainingexamples,Īnd then develop a system which can learn from those trainingexamples. Neural networks approach the problem in a different way. Simple intuitions about how we recognize shapes- "a 9 has a loop at the top, and a vertical stroke in the bottomright" - turn out to be not so simple to express algorithmically.When you try to make such rules precise, you quickly get lost in amorass of exceptions and caveats and special cases. What seems easy when we do it ourselves suddenly becomesextremely difficult. The difficulty of visual pattern recognition becomes apparent if youattempt to write a computer program to recognize digits like thoseabove. And sowe don't usually appreciate how tough a problem our visual systemssolve. But nearly all that work is done unconsciously. Rather, wehumans are stupendously, astoundingly good at making sense of what oureyes show us. Recognizing handwritten digits isn't easy.
And yet human visioninvolves not just V1, but an entire series of visual cortices - V2,V3, V4, and V5 - doing progressively more complex image processing.We carry in our heads a supercomputer, tuned by evolution overhundreds of millions of years, and superbly adapted to understand thevisual world. In each hemisphere of our brain, humans have a primaryvisual cortex, also known as V1, containing 140 million neurons, withtens of billions of connections between them. Most people effortlessly recognize those digits as 504192. Considerthe following sequence of handwritten digits: The human visual system is one of the wonders of the world.
Goodfellow, Yoshua Bengio, and Aaron Courville Michael Nielsen's project announcement mailing list Thanks to all the supporters who made the book possible, withĮspecial thanks to Pavel Dudrenov.
Deep Learning Workstations, Servers, and Laptops