[AI] MNIST Handwritten Digits Recognizer – Single Layered N.N(Neural Network)

Background

I made my first A.I program in this time.

I read a book that is a really good guide to start neural network (N.N).

This book was the most helpful book ever to understand N.N.

I strongly recommend this book, if you are interested in N.N.

And it has little example MNIST recolonization project.

It is implement of single hidden layer N.N model. I used 1568 neurons in hidden layer.

How this works

This program is built to solve MNIST handwritten letter recognize problem.

It read train data as CVS file that provided from internet.

And then query test data to trained network. And it return accuracy of network.

MNIST Data Set

Neural network is huge set of artificial neuron (called as perceptron).

Neuron receive inputs and it return an output if inputs are enough to make output.

If a neuron receive 1,000 of inputs, then multiply each inputs with each weight of input nodes.

And then get sum all of multiplied values.

It will be use to make output, With using activation function (I use sigmoid function in this program. In other ways, there are ReLU, etc.)

We can train neuron by changing weights of inputs. It called back propagation and training.

Back-propagation of errors is processed by how weight is strong.

If two node is connected to a neuron. And it occurred 0.5 error. First weight of node is 1.0 and other is 0.2.

Then node 1’s error is {1.0/(1.0 + 0.5)} * 0.5.

So, nodes gets each’s errors.

And training is processed using differential of activation function.

I cannot understand well training process and differential of activation function.

If you can describe it well, please comment about it!

Video

What’s happening in the video?

Until 3:05 of video is training scene.

You can see ASCIIs(training data, ASCII art is always funny), epoch(how many times that train is repeated), train(answer of training data), percent, and learning per seconds, in this step.

And then user should enter how many count test is processing. In MNIST test database, it contains 10,000 test data records.

You can see real time accuracy (I typed occuracy in this program, not accuracy. it is my typo xD).

This model’s accuracy is about 94%.

Changes from last update

04 April `17 – Optimize memory usage while training; Fix reduced console output.

Source Code

GitHub Repository

Heejun Lee

Hi Everyone! I think we need more active people in CODEX! Everybody, enjoy my codex, and comment!

3 thoughts on “[AI] MNIST Handwritten Digits Recognizer – Single Layered N.N(Neural Network)

Leave a Reply

Your email address will not be published. Required fields are marked *