{"cells":[{"metadata":{},"cell_type":"markdown","source":"#
MNIST handwritten images analysis by numpy.
#\n####
13.12.2019
####\n\n###
Introduction
###\n\n

Handwriting analysis with numpy module.

\n

The data obtained from $Kaggle$ will be used for the analysis.

\n

This jupyret presents handwriting analysis using a multi-layer perceptron neural network built using the numpy module.

\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# connecting packeges\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline\n# Sigmoid\nfrom scipy.special import expit","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Let's create a class for a neural network.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"class NeuralNetwork:\n # initialization function\n def __init__(self, inputnodes, hiddenodes, outputnodes, learningrate):\n # Neuron count\n ## in the incoming layer\n self.inodes = inputnodes\n ## in the hidden layer\n self.hnodes = hiddenodes\n ## in the outgoing layer\n self.onodes = outputnodes\n \n # the weight of the output from the input layer to the hidden layer\n self.wih = np.random.normal(.0, pow(self.inodes, -.5), (self.hnodes, self.inodes))\n # the weight of the output from the hidden layer to the output layer\n self.who = np.random.normal(.0, pow(self.hnodes, -.5), (self.onodes, self.hnodes)) \n \n # Coefficient learning(the accuracy of the model)\n self.lr = learningrate\n \n # # actibation function\n self.activation_function = lambda x: expit(x)\n \n # Error receiving function \n def query(self, inputs_list):\n # Convert to two-dimensional array\n inputs = np.array(inputs_list, ndmin=2).T\n \n # The incoming signal for the hidden layer\n hidden_inputs = np.dot(self.wih, inputs)\n # Outgoing signal for hidden layer\n hidden_outputs = self.activation_function(hidden_inputs)\n \n # The incoming signal for the output layer\n final_inputs = np.dot(self.who, hidden_outputs)\n # Outgoing signal to the output of the hidden layer\n final_outputs = self.activation_function(final_inputs)\n \n return final_outputs\n \n # Training neural network Function\n def train(self, inputs_list, targets_list):\n # Getting output values\n # Convert to two-dimensional array\n inputs = np.array(inputs_list, ndmin=2).T\n \n # The incoming signal for the hidden layer\n hidden_inputs = np.dot(self.wih, inputs)\n # Outgoing signal for hidden layer\n hidden_outputs = self.activation_function(hidden_inputs)\n \n # The incoming signal for the output layer\n final_inputs = np.dot(self.who, hidden_outputs)\n # Outgoing signal to the output of the hidden layer\n final_outputs = self.activation_function(final_inputs)\n \n # Get the desired values \n # Convert to two-dimensional array\n target = np.array(targets_list, ndmin=2).T\n \n # Calculation of the model error\n ## Error in the output layer\n outputs_errors = target - final_outputs\n ## Hidden layer error\n hidden_errors = np.dot(self.who.T, outputs_errors)\n \n \n # Update weights\n ## Between output layer hidden and \n self.who += self.lr * np.dot((outputs_errors * final_outputs * (1 - final_outputs)), np.transpose(hidden_outputs))\n ## Between hidden and input layer\n self.wih += self.lr * np.dot((hidden_errors * hidden_outputs * (1 - hidden_outputs)), np.transpose(inputs)) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Download the necessary data.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/digit-recognizer/train.csv')\ntest = pd.read_csv('../input/digit-recognizer/test.csv')\nsample = pd.read_csv('../input/digit-recognizer/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Let's look at what the data is.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"temp = train.drop(labels = [\"label\"],axis = 1).values.reshape(-1,28,28,1)\nplt.figure(figsize=(15,4.5))\nfor i in range(30): \n plt.subplot(3, 10, i+1)\n plt.imshow(temp[i].reshape((28,28)),cmap=plt.cm.binary)\n plt.axis('off')\nplt.subplots_adjust(wspace=-0.1, hspace=-0.1)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Since the data represents the power of color, the data is distributed from 0 to 255. for better performance, normalize the data from 0 to 1.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"scaled_input = (np.array(train.iloc[1, 1:]) / 255 * .99) + .01","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Let's introduce the necessary neurons. The image is presented in the format of 28X28 pixels. which corresponds to 784 neurons. We will also introduce 100 hidden neurons. Since the output we get numbers from 0 to 9 the number of output neurons is 10.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"inputnodes = 784\nhiddenodes = 100\noutputnodes = 10\nlearningrate = .3\n\n# Create neural network\nnn = NeuralNetwork(inputnodes, hiddenodes, outputnodes, learningrate)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Let's introduce the necessary neurons. The image is presented in the format of 28X28 pixels. which corresponds to 784 neurons. We will also introduce 100 hidden neurons. Since the output we get numbers from 0 to 9 the number of output neurons is 10.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"for pix in train.index:\n values = np.array(train.iloc[pix, 1:])\n inputs = (np.asfarray(values) / 255 * .99) + .01\n targets = np.zeros(outputnodes) + .01\n targets[int(train.iloc[pix, 1])] = .99\n nn.train(inputs, targets)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Let's check the neural network on the test data and find the accuracy of the predicted network.

"},{"metadata":{"trusted":true},"cell_type":"code","source":"scorecard = []\n\nfor pix in test.index:\n values = np.array(test.iloc[pix, :])\n correct_label = int(sample.loc[pix, 'Label'])\n inputs = (np.asfarray(values) / 255 * .99) + .01\n outputs = nn.query(inputs)\n label = np.argmax(outputs)\n if label == correct_label:\n scorecard.append(1)\n else:\n scorecard.append(0)\n# print(correct_label, label, end='||')\nscorecard_array = np.asarray(scorecard)\n# scorecard_array.sum() / scorecard_array.size\nans = scorecard_array.sum() / scorecard_array.size\nprint('The accuracy of the predicted values of the neural network: %.2f%%' % (ans * 100))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"###
Summary:
###\n\n

When using a neural network built on the module numpy bul received a positive result of the analysis of handwriting.

"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.3"}},"nbformat":4,"nbformat_minor":1}