{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n## Table of Contents\n\n1. [Introduction to Artificial Neural Network (ANN)](#1)\n1. [Input Data to a Neural Network](#2)\n1. [MNIST dataset](#3)\n1. [Data representation in Keras](#4)\n1. [Data normalization in Keras](#5)\n1. [Densely connected networks in Keras](#6)\n - 6.1 [Sequential Class in Keras](#6.1)\n - 6.2 [Defining the model](#6.2) \n1. [Model Summary](#7)\n - 7.1 [Parameters of the model](#7.1)\n1. [Implementation of Neural Network in Keras](#8)\n - 8.1 [Compilation of model](#8.1)\n - 8.2 [Training the model](#8.2)\n - 8.3 [Evaluation of model](#8.3)\n1. [Accuracy of the model](#9)\n1. [Confusion-matrix](#10)\n1. [Generate predictions](#11)\n1. [ANN- The Complete Example](#12)\n1. [Conclusion](#13)\n1. [References](#14)\n\n ","metadata":{}},{"cell_type":"markdown","source":"## 1. Introduction to Artificial Neural Network (ANN) \n\n[Back to Table of Contents](#0.1)\n\n\n- An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. \n\n- Each connection, like the synapses in a biological brain, can transmit a signal to other neurons. An artificial neuron that receives a signal then processes it and can signal neurons connected to it.\n\nFor detailed discussion on ANN, please visit:\n\n[Artificial Neural Network](https://en.wikipedia.org/wiki/Artificial_neural_network)\n\n\nFor simplified working of ANN, please vist:\n\n[Working of ANN in simple terms](https://www.analyticsvidhya.com/blog/2014/10/ann-work-simplified/)\n\n","metadata":{}},{"cell_type":"markdown","source":"## 2. Input Data to a Neural Network \n\n[Back to Table of Contents](#0.1)\n\n- For the construction and evaluation of a model in Deep Learning, the available data are divided into three sets:\n**training data**, **validation data** and **test data**.\n\n- The **training data** is used for the learning algorithm to obtain the parameters of the model.\n\n- Now, suppose the overfitting exists. Then, we would modify the value of certain hyperparameters and after training it again with the training data, we would evaluate it again with the **validation data**. We can make these adjustments of the hyperparameters guided by the validation data until we obtain optimum results.\n\n- The validation data have influenced the model so that it also fits the validation data. So, we always reserve a set of **test data** for final evaluation of the model that will only be used at the end of the whole process, when we consider that the model is already fine-tuned and we will no longer modify any of its hyperparameters.","metadata":{}},{"cell_type":"markdown","source":"## 3. MNIST dataset \n\n[Back to Table of Contents](#0.1)\n\n\n- MNIST is a collection of handwritten digits ranging from the number 0 to 9.\n\n- It has a training set of 60,000 images and 10,000 test images that are classified into corresponding categories or labels. \n\n- In Keras the MNIST dataset is preloaded in the form of four Numpy arrays.\n\n- To use the MNIST dataset in Keras, an API is provided to download and extract images and labels automatically.\n\n- The following Keras code shows how to access MNIST dataset, plot 25 random samples, and count the number of labels for train and test datasets:","metadata":{}},{"cell_type":"code","source":"# import necessary libraries\nimport keras\nimport numpy as np\n\n\n# load dataset\nfrom keras.datasets import mnist\n\n\n# split dataset into training and test set\n(x_train, y_train), (x_test, y_test) = mnist.load_data()","metadata":{"execution":{"iopub.status.busy":"2021-09-30T15:58:19.500638Z","iopub.execute_input":"2021-09-30T15:58:19.500976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training and Test set\n\n- `x_train` and `y_train` contain the training set, while\n\n- `x_test` and `y_test` contain the test set.\n\n- The images are encoded as Numpy arrays and their corresponding labels ranging from 0 to 9.","metadata":{}},{"cell_type":"markdown","source":"### Check the values loaded\n\n\n- To check the values that we have loaded, we choose any of the images of the MNIST set, say for example image 7, and write the following Python code:","metadata":{}},{"cell_type":"code","source":"# Display images\n\nimport matplotlib.pyplot as plt\nplt.imshow(x_train[7], cmap=plt.cm.binary)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We get the above image.","metadata":{}},{"cell_type":"markdown","source":"### View the corresponding labels\n\n- To see its corresponding label we can write the following Python code:","metadata":{}},{"cell_type":"code","source":"# View the labels\nprint(y_train[7])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It returns the value of \"3\", as expected.","metadata":{}},{"cell_type":"markdown","source":"## 4. Data representation in Keras \n\n[Back to Table of Contents](#0.1)\n\n- Keras uses a multidimensional array of Numpy as a basic data structure. We call this data structure a **tensor**. \n\n- A tensor has three main attributes:\n\n - **Number of axes (Rank)**: \n \n - A tensor containing a single number will be called **scalar** (or a 0-dimensional tensor, or tensor 0D). \n - An array of numbers is called **vector**, or tensor 1D. \n - An array of vectors will be a matrix, or 2D tensor. \n - If we pack this matrix in a new array, we get a 3D tensor, which we can interpret visually as a cube of numbers. \n - By packaging a 3D tensioner in an array, we can create a 4D tensioner and so on. \n - In the Python Numpy library this is called the **tensor’s ndim**.\n \n \n - **Shape**:\n \n - It is a tuple of integers that describe how many dimensions the tensor has along each axis. \n - In the Numpy library this attribute is called **Shape**.\n \n \n - **Data type**: \n \n - This attribute indicates the type of data that contains the tensor, which can be for example uint8, float32, float64, etc. \n - In the Numpy library this attribute is called **dtype**.\n\n\n","metadata":{}},{"cell_type":"markdown","source":"We can obtain the number of axes and dimensions of the tensor `train_images` from our previous example as follows:","metadata":{}},{"cell_type":"code","source":"# View number of dimensions of tensor\n\nprint(x_train.ndim)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View the dimension of tensor\n\nprint(x_train.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To know the data type it contains, we can run the following command:","metadata":{}},{"cell_type":"code","source":"# View the data type of tensor\n\nprint(x_train.dtype)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Data normalization in Keras \n\n[Back to Table of Contents](#0.1)\n\n\n- The MNIST images of 28×28 pixels are represented as an array of numbers whose values range from [0, 255] of type uint8. \n- It is usual to scale the input values of neural networks to certain ranges. \n- In this example, the input values should be scaled to values of type float32 within the interval [0, 1]. \n- This can be done with the following lines of Python code:","metadata":{}},{"cell_type":"code","source":"# scale the input values to type float32\n\nx_train = x_train.astype('float32')\nx_test = x_test.astype('float32')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# scale the input values within the interval [0, 1]\n\nx_train /= 255\nx_test /= 255","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- To facilitate the entry of data into our neural network we must make a transformation of the tensor (image) from 2 dimensions (2D) to a vector of 1 dimension (1D). \n- That is, the matrix of 28×28 numbers can be represented by a vector (array) of 784 numbers (concatenating row by row), which is the format that accepts as input a densely connected neural network. \n- In Python, converting every image of the MNIST dataset to a vector with 784 components can be accomplished as follows:","metadata":{}},{"cell_type":"code","source":"x_train = x_train.reshape(60000, 784)\nx_test = x_test.reshape(10000, 784)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- After executing the above Python instructions, we can verify that `x_train.shape` gives (60000, 784) and `x_test.shape` gives (10000, 784). \n- The first dimension indexes the image and the second indexes the pixel in each image (now the intensity of the pixel is a value between 0 and 1):","metadata":{}},{"cell_type":"code","source":"print(x_train.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(x_test.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Now we have the labels for each input data. \n- They are numbers between 0 and 9 that indicate which digit represents the image, that is, to which class they are associated. \n- We will represent this label with a vector of 10 positions, where the position corresponding to the digit that represents the image contains a 1 and the remaining positions of the vector contain the value 0.","metadata":{}},{"cell_type":"markdown","source":"- So, we will use the **one-hot encoding** procedure. \n- It consists of transforming the labels into a vector of as many zeros as the number of different labels, and containing the value of 1 in the index that corresponds to the value of the label. \n- Keras offers many support functions, including `to_categorical` to perform this transformation, which we can import from `keras.utils`:","metadata":{}},{"cell_type":"code","source":"from keras.utils import to_categorical","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- To see the effect of the transformation we can see the values before and after applying `to_categorical`:","metadata":{}},{"cell_type":"code","source":"print(y_test[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_train[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_train.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(x_test.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train = to_categorical(y_train, num_classes=10)\n\ny_test = to_categorical(y_test, num_classes=10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_test[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_train[0])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_train.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(y_test.shape)\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Now, our dataset is ready to be fed in the neural network.\n- We will program it in Keras in the following section.","metadata":{}},{"cell_type":"markdown","source":"## 6. Densely connected networks in Keras \n\n\n[Back to Table of Contents](#0.1)\n\n\n- Now, we will start building the fully connected network in Keras.\n\n- I will present how to specify the model in Keras.","metadata":{}},{"cell_type":"markdown","source":"### 6.1 Sequential class in Keras \n\n\n- The main data structure in Keras is the Sequential class, which allows the creation of a basic neural network.\n\n- The Sequential class of the Keras library is a wrapper for the sequential neural network model that Keras offers and can be created in the following way:\n\n\n`from keras.models import Sequential`\n\n`model = Sequential()`\n\n- The model in Keras is considered as a sequence of layers and each of them gradually “distills” the input data to obtain the desired output. \n\n- In Keras, we can add the required types of layers through the **.add()** method.","metadata":{}},{"cell_type":"markdown","source":"### 6.2 Defining the model \n\n- We can program the model in Keras to recognize the images of digits in the following way:","metadata":{}},{"cell_type":"code","source":"from keras.models import Sequential\nfrom keras.layers.core import Dense, Activation\n\n\nmodel = Sequential()\nmodel.add(Dense(10, activation='sigmoid', input_shape=(784,)))\nmodel.add(Dense(10, activation='softmax'))\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Here, the neural network has been defined as a sequence of two layers that are densely connected (or fully connected).\n\n- It means that all the neurons in each layer are connected to all the neurons in the next layer.","metadata":{}},{"cell_type":"markdown","source":"#### First layer\n\n- We explicitly express in the `input_shape` argument of the first layer what the input data is like: a tensor that indicates that we have 784 features of the model.\n\n- The tensor is being defined is (None, 784,).","metadata":{}},{"cell_type":"markdown","source":"#### Second layer\n\n- The second layer is a softmax layer of 10 neurons, which means that it will return a matrix of 10 probability values representing the 10 possible digits. \n\n- Each value will be the probability that the image of the current digit belongs to each one of them.","metadata":{}},{"cell_type":"markdown","source":"## 7. Model Summary \n\n[Back to Table of Contents](#0.1)\n\n\n- Keras provides a very useful method to check the architecture of the model.\n\n- It is the **.summary()** method.\n\n- We can use this **.summary()** method as follows:","metadata":{}},{"cell_type":"code","source":"model.summary()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 7.1 Parameters of the model \n\n- For our simple example, we see that it indicates that 7,960 parameters are required (column Param #), which correspond to 7850 parameters to the first layer and 110 to the second.\n\n\n#### Parameters of the first layer\n\n- In the first layer, for each neuron i (between 0 and 9) we require 784 parameters for the weights wij and therefore 10×784=7840 parameters to store the weights of the 10 neurons. \n\n- Also, 10 additional parameters for the 10 bj biases corresponding to each one of them is required. \n\n- So, for the first layer we require 7840 + 10 = 7850 parameters.\n\n\n#### Parameters of the second layer\n\n- In the second layer, being a softmax function, it is required to connect all 10 neurons with the 10 neurons of the previous layer. \n\n- Therefore 10x10=100 wi parameters are required and in addition 10 bj biases corresponding to each node is required.\n\n- So, we require 100 + 10 = 110 parameters for the second layer.","metadata":{}},{"cell_type":"markdown","source":"## 8. Implementation of Neural Network in Keras \n\n[Back to Table of Contents](#0.1)\n\n\n- The implementation of Neural Network in Keras comprises of three steps:-\n\n - Compiling the model with the **compile()** method.\n - Training the model with **fit()** method.\n - Evaluating the model with **evaluate()** method.\n \n \n- These steps are as described below.\n","metadata":{}},{"cell_type":"markdown","source":"### 8.1 Compile model with **compile()** method \n\n- Starting with the Sequential model, we can define the layers in a simple way with the **add()** method.\n\n- The next step is to compile the model with the **compile()** method, with which we can specify some properties through method arguments which are as follows:\n\n#### Loss Function\n\n- The first argument is the **loss function**.\n\n- We will use it to evaluate the degree of error between calculated outputs and the desired outputs of the training data. \n\n#### Optimizer\n\n- The second argument is the **optimizer**.\n\n- It is the way we have to specify the optimization algorithm that allows the neural network to calculate the weights of the parameters from the input data and the defined loss function. \n\n#### Metrics\n\n- The third argument is the **metrics**.\n\n- We must indicate the metric that we will use to monitor the learning process and test of our neural network. \n\n- In this example, we will only consider the accuracy - the fraction of images that are correctly classified.","metadata":{}},{"cell_type":"markdown","source":"- In this example we specify that the -\n\n - loss function is **categorical_crossentropy**, \n - the optimizer used is **the stocastic gradient descent (sgd)**, and \n - the metric is **accuracy**, with which we will evaluate the percentage of correct guesses.","metadata":{}},{"cell_type":"code","source":"model.compile(loss=\"categorical_crossentropy\",\n optimizer=\"sgd\",\n metrics = ['accuracy'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 8.2 Train the model with fit() method \n\n- Now, that we have compiled our model, the next step is to train the model.\n\n- We can train the model with the **fit()** method as follows:","metadata":{}},{"cell_type":"code","source":"model.fit(x_train, y_train, batch_size=100, epochs=10)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- In the first two arguments **(x_train, y_train)** we have indicated the data with which we will train the model in the form of Numpy arrays. \n\n- The **batch_size** argument indicates the number of data that we will use for each update of the model parameters.\n\n- The **epochs** argument indicate the number of times we will use all the data in the learning process.","metadata":{}},{"cell_type":"markdown","source":"### 8.3 Evaluate model with evaluate() method \n\n\n- Our model has been compiled and trained.\n\n- Now, we come to the final step.\n\n- The model can now be evaluated with the **evaluate()** method as follows:\n\n","metadata":{}},{"cell_type":"code","source":"test_loss, test_acc = model.evaluate(x_test, y_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 9. Accuracy of the model \n\n[Back to Table of Contents](#0.1)\n\n- Now, I will check the accuracy of the model.\n\n- We can print the accuracy as follows:-\n\n","metadata":{}},{"cell_type":"code","source":"print('Test accuracy:', round(test_acc,4))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 10. Confusion-matrix \n\n[Back to Table of Contents](#0.1)\n\n\n- In Machine Learning, a very useful tool to evaluate the model is **Confusion Matrix**.\n\n- It is a table with rows and columns that count the predictions in comparison with the real values. \n\n- This table can be used to understand the performance of the model. \n\n- It is very useful to show explicitly when one class is confused with another. \n\n- A confusion matrix for a binary classifier is shown below:-","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"- **True Positives (TP)**, **True Negatives (TN)**, **False Positives (FP)** and **False Negatives (FN)** are the four different possible outcomes of a single prediction for a two-class case with classes “1” (“positive”) and “0” (“negative”).\n\n- A **False Positive (FP)** is when the outcome is incorrectly classified as positive, when it is in fact negative. A **False Negative (FN)** is when the outcome is incorrectly classified as negative when it is in fact positive. **True Positives (TP)** and **True Negatives (TN)** are obviously correct classifications.\n\n- With this confusion matrix, the accuracy can be calculated by adding the values of the diagonal and dividing them by the total:\n\n**Accuracy = (TP + TN) / (TP + FP + FN + TN)**\n\n\n- There is another metric called **Sensitivity (or recall)** that tells us how well the model avoids false negatives:\n\n**Sensitivity = TP / (TP + FN)**\n\n\n- From the confusion matrix, several other metrics can be obtained.\n\n- But in this case, there are 10 classes instead of 2. So, in this case, Scikit-learn package can be used to evaluate the quality of the model by calculating the confusion matrix.\n\n- The following code can be used to generate the confusion-matrix:","metadata":{}},{"cell_type":"code","source":"# Plot confusion matrix \n# Note: This code snippet for confusion-matrix is taken directly from the SKLEARN website.\ndef plot_confusion_matrix(cm, classes,\n normalize=False,\n title='Confusion matrix',\n cmap=plt.cm.Blues):\n \"\"\"\n This function prints and plots the confusion matrix.\n Normalization can be applied by setting `normalize=True`.\n \"\"\"\n plt.imshow(cm, interpolation='nearest', cmap=cmap)\n plt.title(title)\n plt.colorbar()\n tick_marks = np.arange(len(classes))\n plt.xticks(tick_marks, classes, rotation=30)\n plt.yticks(tick_marks, classes)\n\n if normalize:\n cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n\n thresh = cm.max() / 2.\n for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n plt.text(j, i, cm[i, j],\n horizontalalignment=\"center\",\n color=\"white\" if cm[i, j] > thresh else \"black\")\n\n plt.tight_layout()\n plt.ylabel('Actual class')\n plt.xlabel('Predicted class')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from collections import Counter\nfrom sklearn.metrics import confusion_matrix\nimport itertools\n\n# Predict the values from the validation dataset\nY_pred = model.predict(x_test)\n# Convert predictions classes to one hot vectors \nY_pred_classes = np.argmax(Y_pred, axis = 1) \n# Convert validation observations to one hot vectors\nY_true = np.argmax(y_test, axis = 1) \n# compute the confusion matrix\nconfusion_mtx = confusion_matrix(Y_true, Y_pred_classes) \n# plot the confusion matrix\nplot_confusion_matrix(confusion_mtx, classes = range(10))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- In this case, the elements of the diagonal represent the number of points in which the label predicted by the model coincides with the actual value of the label, while the other values indicate the cases in which the model has classified incorrectly. \n\n- Therefore, the higher the values of the diagonal, the better the prediction will be. \n\n- If we calculate the sum of the values of the diagonal divided by the total values of the matrix, we get the same accuracy that the evaluate() method has returned.","metadata":{}},{"cell_type":"markdown","source":"## 11. Generate predictions \n\n[Back to Table of Contents](#0.1)\n\n\n- We have build the ANN model to classify the MNIST digits.\n\n- Now, we need to know how we can use the trained model to make predictions. \n\n- It consists in predict which digit represents an image. \n\n- In order to do this, Keras supply the **predict()** method.\n\n- To test this method we can choose any element. \n\n- For simplicity, we will take one from the test dataset x_test. For example let’s choose the element 11 of this dataset x_test.","metadata":{"trusted":true}},{"cell_type":"markdown","source":"- Now, we will see the **predict()** method in action. \n\n- We execute the following code, correctly predicts the value that it should predict.","metadata":{}},{"cell_type":"code","source":"predictions = model.predict(x_test)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- The predict() method return a vector with the predictions for the whole dataset elements. \n\n- We know which class gives the most probability of belonging by means of the argmax function of Numpy, which returns the index of the position that contains the highest value of the vector. \n\n- Specifically, for item 11:","metadata":{}},{"cell_type":"code","source":"np.argmax(predictions[11])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can check it printing the vector returned by the method:","metadata":{}},{"cell_type":"code","source":"print(predictions[11])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- We see that the highest value in the vector is in the position 6. \n\n- We can also verify that the result of the prediction is a vector whose sum of all its components is equal to 1, as expected. \n\n- For this we can use:","metadata":{}},{"cell_type":"code","source":"np.sum(predictions[11])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 12.ANN - The complete example \n\n[Back to Table of Contents](#0.1)\n\n- In this section, I will combine all the pieces together and write the complete ANN model for convenience.\n","metadata":{}},{"cell_type":"code","source":"### Get the basics\n\n# import Keras library\n#import keras\n\n\n# load dataset\n# from keras.datasets import mnist\n\n\n# split dataset into training and test set\n# (x_train, y_train), (x_test, y_test) = mnist.load_data()\n\n#############################################################################\n\n### Display images\n\n# import matplotlib.pyplot as plt\n\n# plt.imshow(x_train[7], cmap=plt.cm.binary)\n\n# View the labels\n\n# print(y_train[7])\n\n#############################################################################\n\n### Data representation in Keras\n\n\n# View number of dimensions of tensor\n\n# print(x_train.ndim)\n\n\n\n# View the dimension of tensor\n\n# print(x_train.shape)\n\n\n\n# View the data type of tensor\n\n# print(x_train.dtype)\n\n\n##############################################################################\n\n### Data normalization in Keras\n\n\n# Scale the input values to type float32\n\n# x_train = x_train.astype('float32')\n# x_test = x_test.astype('float32')\n\n\n# Scale the input values within the interval [0, 1]\n# x_train /= 255\n# x_test /= 255\n\n\n# Reshape the input values\n# x_train = x_train.reshape(60000, 784)\n# x_test = x_test.reshape(10000, 784)\n\n# print(x_train.shape)\n# print(x_test.shape)\n\n\n# from keras.utils import to_categorical\n\n# print(y_test[0])\n# print(y_train[0])\n\n# print(y_train.shape)\n# print(x_test.shape)\n\n# y_train = to_categorical(y_train, num_classes=10)\n# y_test = to_categorical(y_test, num_classes=10)\n\n# print(y_test[0])\n# print(y_train[0])\n\n# print(y_train.shape)\n# print(y_test.shape)\n\n###############################################################################\n\n### Define the model\n\n# from keras.models import Sequential\n# from keras.layers.core import Dense, Activation\n\n# model = Sequential()\n# model.add(Dense(10, activation='sigmoid', input_shape=(784,)))\n# model.add(Dense(10, activation='softmax'))\n\n###############################################################################\n\n### Model summary\n\n# model.summary()\n\n###############################################################################\n\n### Implementation of Neural Network in Keras\n\n\n# Compiling the model with compile() method\n\n# model.compile(loss=\"categorical_crossentropy\",\n# optimizer=\"sgd\", metrics = ['accuracy'])\t\n\n\n# Training the model with fit() method\n\n# model.fit(x_train, y_train, batch_size=100, epochs=10)\n\n\n# Evaluate model with evaluate() method\n\n# test_loss, test_acc = model.evaluate(x_test, y_test)\n\n\n################################################################################\n\n### Accuracy of the model\n\n# print('Test accuracy:', round(test_acc,4))\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 13. Conclusion \n\n[Back to Table of Contents](#0.1)\n\n- In this kernel, I build a simple ANN model to classify the MNIST digits and predict accuracy.\n\n- We get the test accuracy of 82.57%.\n\n- The accuracy tells us that our ANN model classifies the digits 82.57% of time correctly.","metadata":{}},{"cell_type":"markdown","source":"## 14. References \n\n\n[Back to Table of Contents](#0.1)\n\n\n- The concepts and ideas in this kernel are taken from the following books:\n\n - 1. Deep Learning with Python by Francois Chollet.\n\n - 2. Advanced Deep Learning with Keras by Rowel Atienza","metadata":{}},{"cell_type":"markdown","source":"- So, now we will come to the end of this kernel. \n\n- In the next kernel, we will demonstrate how we can improve these classification results using convolutional neural networks (CNN) for the same MNIST digits example.","metadata":{}},{"cell_type":"markdown","source":"I hope you find this kernel useful and enjoyable.\n\nThank you\n","metadata":{}},{"cell_type":"markdown","source":"[Go to Top](#0)","metadata":{}}]}