{"cells":[{"metadata":{"_uuid":"5507da4ce26bdea823abbd83bfa2d8827186b75a"},"cell_type":"markdown","source":"\n
Content:\n* [Introduction](#1)\n* [Aim](#1-1)\n* [Overview the Data Set](#2) \n* [Import Required Pyton Libraries](#3 )\n* [ Loading The Dataset ](#4)\n * [Loading Train Raw Dataset and Detail](#4-1)\n * [Loading Test Raw Dataset and Detail](#4-2)\n * [Check data for any null values](#4-3)\n* [Data Visualization](#5)\n * [ Drawing Sample of the Testing Data](#5)\n * [Drawing Train Data With Label](#5-1)\n* [Data Preparation](#6)\n * [ Data Normalization](#6-1)\n * [One Hot encoding of labels](#6-2) \n* [Initialization the random number generator](#7-1)\n* [Designing Multilayer Perceptron-MLP Architecture](#7)\n * [Activation function](#7-2) \n * [Create model](#7-3)\n * [Save model](#7-4) \n * [Model Drawing to visualize the behavior](#7-5)\n * [Model Score](#7-6) \n * [Model-1](#7-7)\n * [Model-2](#7-8)\n * [Model-3](#7-10)\n * [Compare Model-3 to Model-2](#7-11)\n * [Model-4](#7-12) \n * [Model-5](#7-14) \n "},{"metadata":{"_uuid":"de54b9ff8a74e0c82dbb59dc3deabee0c07428ef"},"cell_type":"markdown","source":"
\n# INTRODUCTION\n* **Deep learning:** One of the machine learning technique that learns features directly from data. \n* **Why deep learning:** When the amounth of data is increased, machine learning techniques are insufficient in terms of performance and deep learning gives better performance like accuracy.\n\"1\"\n* **What is amounth of big:** It is hard to answer but intuitively 1 million sample is enough to say \"big amounth of data\"\n* **Usage fields of deep learning:** Speech recognition, image classification, natural language procession (nlp) or recommendation systems\n* **What is difference of deep learning from machine learning:** \n * Machine learning covers deep learning. \n * Features are given machine learning manually.\n * On the other hand, deep learning learns features directly from data.\n\"2\"\n\n"},{"metadata":{"_uuid":"098d5a08694a99f7567a1351b077bc2c503f2d92"},"cell_type":"markdown","source":"
\n# AIM\n Multi classification on Hand writing images database. We will use deep learning model. This model is MLP-Multilayer Perceptron."},{"metadata":{"_uuid":"de1d57fddcd736229a6775e2c78c4327dd528fd9"},"cell_type":"markdown","source":"
\n# Overview The Data Set\n* We will use \"MNIST in CSV\".\n* In this data there are 60000 digits images. 0 to 59999 index\n* As you know digits are from 0 to 9. "},{"metadata":{"_uuid":"b43a4a0a09642e12334754f006a52cf6265fcee8"},"cell_type":"markdown","source":" \n# Import required Pyton libraries\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns # visualization tool\n# keras imports for the dataset and building our neural network\nfrom keras.datasets import mnist\nfrom keras.preprocessing import image\nfrom keras.utils import to_categorical\nfrom keras.models import Sequential, load_model\nfrom keras.layers.core import Dense, Dropout, Activation\nfrom keras.utils import np_utils\n\nfrom keras.optimizers import Adam\nfrom keras.callbacks import ModelCheckpoint\nimport tensorflow as tf\n\nseed = 42\nnp.random.seed(seed)\n\nfrom collections import Counter\nimport itertools\nimport os\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nfrom subprocess import check_output\nprint('Input Folder Detail :' )\nprint('----------------------' )\nprint(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7f14db23ba4ac4fa2ffbf02d9166e6f0c7407be0"},"cell_type":"markdown","source":"
\n# Loading The Dataset \n \n## Train Raw Database and Detail"},{"metadata":{"trusted":true,"_uuid":"72352da5cdf8d3fff7876249dde5d9fbd8672513","scrolled":true},"cell_type":"code","source":"# Reading the data frames\n#Training Data\ntrain_raw = pd.read_csv(\"../input/mnist_train.csv\") \n# data info\nprint('')\nprint('Train Data Info :' )\nprint('------------------' )\ntrain_raw.info()\nprint('')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"37f0bba4f160c6fa14ecb16442db3cc163dddc4a"},"cell_type":"code","source":"print('data_train shape:', train_raw.shape)\nprint('Number of images in train :', train_raw.shape[0])\ntrain_raw.head() \n#The Head Table = 5 rows × 785 columns \n#The Row = 784 features, 1 label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"51e205d7487ce7cfca8d4bc70ce57545153fb194"},"cell_type":"code","source":"z_train = Counter(train_raw['label'])\nprint(z_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"452e9fa2bbb48e843354caa62be159fadb42a76a"},"cell_type":"code","source":"train_raw.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"21a012175b4225d308824c3c2c993de195cb63b4"},"cell_type":"code","source":"# Countplot for labels\nsns.countplot(train_raw['label'])\nprint(\"Dataset is pretty balanced!\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"33c03176e16dfccbfd6b346ebc8920d18a56b1fc"},"cell_type":"markdown","source":"
\n# Loading The Dataset \n## Test Database and Detail"},{"metadata":{"trusted":true,"_uuid":"3b01bfccfbec027bdee1c16dff8c1e99fe14af66","scrolled":true},"cell_type":"code","source":"# Reading the data frames\n#Test Data\ntest_raw = pd.read_csv(\"../input/mnist_test.csv\") \n# data info\nprint('')\nprint('Test Data Info :' )\nprint('------------------' )\ntest_raw.info()\nprint('')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1d9f379601e034dd12c00fbf606ca3d8d201405d"},"cell_type":"code","source":"print('data_test shape:', test_raw.shape)\nprint('Number of images in data_test :', test_raw.shape[0])\ntest_raw.head()\n# 5 rows × 785 columns\n# 784 features, 1 label","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8e3bec80dd78b9871f0418de8a75a4dd3fc88e36"},"cell_type":"markdown","source":"
\n# Check data for any null values\n* Check data for any null values\n* Is the dataset clean ?"},{"metadata":{"trusted":true,"_uuid":"819e5f44c5f603311d36e123acae4f6847613a17"},"cell_type":"code","source":"train_raw.isnull().any().describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f716ef0480e215c07ec4a9eab9ef22e7c8294978"},"cell_type":"code","source":"test_raw.isnull().any().describe()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1ffb04353cfed753b1686b6a0a34f5b5e65e8f76"},"cell_type":"markdown","source":"There is no null values in the dataset."},{"metadata":{"trusted":true,"_uuid":"7ab8cb2a7179f4319d956ed45d8f8d0d81ab4a86"},"cell_type":"code","source":"# Assign features and corresponding labels\n#Converting the Pandas frame to numpy \n\n#Train data\nX_train = train_raw.iloc[:,1:].values.astype('float32') # all pixel values in train\ny_train_label = train_raw.iloc[:,0].values.astype('int32') # only train labels i.e targets digits\n\n#Test data\nX_test = test_raw.iloc[:,1:].values.astype('float32') # or np.array(test.iloc[:,1:],np.float32) # all pixel values in test\ny_test_label = test_raw.iloc[:,0].values.astype('int32') # only test labels i.e targets digits\n\n#X_train = train_raw.iloc[:, 1:785]\n#train_label = train_raw.iloc[:, 0]\n#X_test = test_raw.iloc[:, 1:785]\n#X_test = test_raw.iloc[:, 1:785]\n\n#label values == ['0','1','2','3','4','5','6','7','8','9']\nlabel = np.array(train_raw.iloc[:,0],np.str) #or label = train_raw.iloc[:,0].values.astype('str')\n\nprint('X_train:', X_train)\nprint('X_train_shape:', X_train.shape)\nprint('X_test.shape:',X_test.shape)\nprint('X_test:', X_test)\nprint('y_train_label:', y_train_label)\nprint('y_train_label.shape:',y_train_label.shape)\nprint('y_test_label:', y_test_label)\nprint('y_test_label.shape:',y_test_label.shape)\nprint('Label:',label)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4fc955ee470c2ababb2d48e6a0ffd765de7d60e0"},"cell_type":"markdown","source":"
\n# Data Visualization\n## Drawing Sample of the Testing Data"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"def visualize_input(img, ax):\n ax.imshow(img, cmap='gray')\n width, height = img.shape\n thresh = img.max()/2.5\n for x in range(width):\n for y in range(height):\n ax.annotate(str(round(img[x][y],2)), xy=(y,x),\n horizontalalignment='center',\n verticalalignment='center',\n color='white' if img[x][y] \n# Data Visualization\n## Drawing Train Data With Label\nLets look at 14 images from data set with their labels."},{"metadata":{"trusted":true,"_uuid":"ff591456df4b30ba1568e3e223acc1dac270524c"},"cell_type":"code","source":"#Drawing Train Data\nfig = plt.figure(figsize=(20,20))\nfor i in range(14):\n ax = fig.add_subplot(7,7,i+1)\n ax.imshow(np.reshape(X_train[i],(28,28)),cmap='gray')\n plt.tight_layout()\n ax.set_title(\"Digit: {}\".format(label[i]))\n #ax.set_title(str(label[i]))\n#fig ","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"3dcd5480821ed77a54768c9cf8dc687df9d59200"},"cell_type":"markdown","source":"
\n# Data Preparation\n* The dataset is clean no need for cleaning\n\n"},{"metadata":{"trusted":true,"_uuid":"a2cc6165935ad576eeaa9458ecf204ba694678a0"},"cell_type":"code","source":"# Making sure that the values are float so that we can get decimal points after division\nX_train = X_train.reshape(60000, 784)\nX_test = X_test.reshape(10000, 784)\nX_train = X_train.astype('float32')\nX_test = X_test.astype('float32')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f640c902a4f862e250873fd3096f0cbb2ccaf01c"},"cell_type":"markdown","source":"
\n# Data Normalization\n## We have to normalize our data, in the case of the MNIST dataset, this simply means dividing by 255, a more formal explanations is that \n\n### $$X = \\frac{X - min(x)}{max(x) - min(x)}$$\n### $$X = \\frac{X - 0}{255 - 0}$$$$X = \\frac{X }{255 }$$\n* Normalizing the data as each pixel is from 0 : 255 => Black => white in gray scale\n* Normalizing the RGB codes by dividing it to the max RGB value(255).\n* The data will be from 0 to 1"},{"metadata":{"trusted":true,"_uuid":"0ac3860d9b94c1d439dcc1762ab8b87db57107d0"},"cell_type":"code","source":"# Normalizing the RGB codes by dividing it to the max RGB value(255).\n#print((min(X_train[1]), max(X_train[1])))\nX_train = X_train / 255\nX_test = X_test / 255\n\nprint('X_train matrix shape:', X_train.shape) #_train shape: (42000, 28, 28, 1)\nprint('Number of images in X_train :', X_train.shape[0]) #Number of images in X_train : 42000\nprint('X_test matrix shape:', X_test.shape) #X_test shape: (28000, 28, 28, 1)\nprint('Number of images in X_test :', X_test.shape[0])#Number of images in X_test : 28000","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3e6fafc901d963477e26e64074e1afa77c6da69c"},"cell_type":"markdown","source":"
\n# One Hot Encoding of Labels\nA one-hot vector is a vector which is 0 in most dimensions, and 1 in a single dimension. In this case, the nth digit will be represented as a vector which is 1 in the nth dimension.\n\nExample,\n* 0 would be [1,0,0,0,0,0,0,0,0,0].\n* 1 would be [0,1,0,0,0,0,0,0,0,0].\n* 2 would be [0,0,1,0,0,0,0,0,0,0].\n* ............\n* 9 would be [0,0,0,0,0,0,0,0,0,1]."},{"metadata":{"trusted":true,"_uuid":"f408ce05dbb062ba2b013f075af8e6f0fb15057d"},"cell_type":"code","source":"# one-hot encoding using keras' numpy-related utilities\nfrom keras.utils import np_utils\nnum_classes = 10\n\nprint(\"Shape before one-hot encoding Y_train: \", y_train_label.shape)\nprint(\"Shape before one-hot encoding Y_test: \", y_test_label.shape)\nY_train = np_utils.to_categorical(y_train_label, num_classes)\nY_test = np_utils.to_categorical(y_test_label, num_classes)\nprint(\"Shape after one-hot encoding Y_train: \", Y_train.shape)\nprint(\"Shape after one-hot encoding Y_test: \", Y_test.shape)\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d8ee5f08648ac184d94b3c9e51dc45567b672d5e"},"cell_type":"markdown","source":"Plot second label."},{"metadata":{"trusted":true,"_uuid":"21474cbc9fe626f3bbd20e16839f0562d3c7efd5"},"cell_type":"code","source":"plt.title(Y_train[5])\nplt.plot(Y_train[5])\nplt.xticks(range(10));","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"254b61122b21c268870158261ebd72842d020031"},"cell_type":"markdown","source":"\n## Initialization the random number generator\nNext, we can initialize the random number generator to ensure that we always get the same results \nwhen executing this code. This will help if we are debugging."},{"metadata":{"trusted":true,"_uuid":"694d8f0c24153558ed29cdfceabc8759ef6cfecf"},"cell_type":"code","source":"#import numpy as np # linear algebra\n# fix random seed for reproducibility\n#seed = 42\n#np.random.seed(seed)\n#tf.set_random_seed(42) # for tensorflow","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bd64d66e1d3cd3412f970a55a4e907947b1131bd"},"cell_type":"markdown","source":"
\n# Designing Multilayer Perceptron-MLP Architecture\n* Dropout parameter\n* Denses\n* Activation function"},{"metadata":{"_uuid":"e0347648af3f838f94d4b82b0f7537ceb831452a"},"cell_type":"markdown","source":"\n# Activation function\n## The sigmoid function is defined as \n # $$sigmoid(z) = \\frac{1}{1 + e ^ {-z}}$$\n## The relu function is much simpler, as it is the maximum between the input and zero\n## The tanh function is defined as \n# $$tanh(z) = \\frac{e^z - e^{-z}}{e^z + e^{-z}}$$"},{"metadata":{"trusted":true,"_uuid":"5492b7c74f7370c8ff97f05debac6f3e6763d08e"},"cell_type":"code","source":"#python simple code\ndef sigmoid(z):\n output = 1 / (1+np.exp(-z))\n return output\n\ndef relu(z):\n return np.maximum(z, 0)\n\ndef tanh(z):\n return (np.exp(z) - np.exp(-z)) / (np.exp(z) + np.exp(-z))\n \n#plot all activation function\nimport matplotlib.pylab as plt\ndef plot_function(function, title=\"sigmoid\"):\n x = np.arange(-7, 7, 0.01)\n y = function(x)\n \n plt.plot(x, y)\n plt.xlabel(\"x\")\n plt.ylabel(\"y\")\n plt.title(title)\n plt.show()\n \nplot_function(sigmoid, \"sigmoid\") \nplot_function(tanh, \"tanh\")\nplot_function(relu, \"relu\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a6a0a7e024b40c73b6d43e1b9b7f41c74e0ddc86","trusted":true},"cell_type":"code","source":"from keras.models import Sequential\nfrom keras.layers.core import Lambda , Dense, Flatten, Dropout\nfrom keras.callbacks import EarlyStopping\nfrom keras.layers import BatchNormalization, Convolution2D , MaxPooling2D\nfrom keras.optimizers import Adam\nfrom keras.callbacks import ModelCheckpoint\nimport os","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0d4f116a598066a674c83f6cd1e34398f41af893"},"cell_type":"markdown","source":"\n# Create model "},{"metadata":{"trusted":true,"_uuid":"9bde197e09e8f06f9d667c8fafd73a9baaca0f82"},"cell_type":"code","source":"def create_model(input_length, hidden_lenght1, hidden_lenght2, activation_func , \n dropout_val,lr_val,num_epochs, batch_size):\n\n # training the model and saving metrics in history\n model = Sequential()\n\n model.add(Dense(hidden_lenght1, input_dim=input_length, activation=activation_func))\n model.add(Dropout(dropout_val))\n\n model.add(Dense(hidden_lenght2, activation=activation_func))\n model.add(Dropout(dropout_val))\n model.add(Dense(10, activation='softmax'))\n\n lr = lr_val#.001\n adam0 = Adam(lr=lr)\n\n # Modeli derleyip ve daha iyi bir sonuç elde edildiğinde ağırlıkları kaydedelim\n model.compile(loss='categorical_crossentropy', optimizer=adam0, metrics=['accuracy'])\n filepath = 'tmp_weights.best.hdf5'\n checkpoint = ModelCheckpoint(filepath, monitor='acc', verbose=1, save_best_only=True, mode='max')\n callbacks_list = [checkpoint]\n\n # training the model and saving metrics in history\n history_model = model.fit(X_train, Y_train, callbacks=callbacks_list,\n epochs=num_epochs, batch_size=batch_size, verbose=2,\n validation_data = (X_test, Y_test)) # verbose 2 or 0 \n return model, history_model\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"94830b17a27ebbe1128bd01e63e8d413ab73cca5"},"cell_type":"markdown","source":"\n# Save model "},{"metadata":{"trusted":true,"_uuid":"84e3ee3021ba2414d2912604e48615c77dad14f2"},"cell_type":"code","source":"def saving_the_model(model, model_name = 'model_history.h5'): \n model.save(model_name)\n print('Saved trained model at %s ' % model_name)\n ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bd952e27e06de4ea89493087b768c093335888f9"},"cell_type":"markdown","source":"\n# Model Drawing to visualize the behavior "},{"metadata":{"trusted":true,"_uuid":"39b3de3ac95ea869c31afe9fcc898e6793f0b0f7"},"cell_type":"code","source":"def draw_model(training):\n plt.figure()\n plt.subplot(2,1,1)\n plt.plot(training.history['acc'])\n plt.plot(training.history['val_acc'])\n plt.title('model accuracy')\n plt.ylabel('accuracy')\n plt.xlabel('epoch')\n plt.legend(['train', 'test'], loc='lower right')\n\n plt.subplot(2,1,2)\n plt.plot(training.history['loss'])\n plt.plot(training.history['val_loss'])\n plt.title('model loss')\n plt.ylabel('loss')\n plt.xlabel('epoch')\n plt.legend(['train', 'test'], loc='upper right')\n\n plt.tight_layout()\n plt.show() \n ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"264af5dff0510de6615eb62f5362d48018bae5dc"},"cell_type":"markdown","source":"\n# Model Score"},{"metadata":{"trusted":true,"_uuid":"6495036dcd1ae468b001d095268a65078deb7e6c"},"cell_type":"code","source":"# evaluate test accuracy\ndef scoring_the_model(model_name=\"model_history.h5\"):\n # load the model and create predictions on the test set\n mnist_model = load_model(model_name)\n #1- genel bakış\n loss_and_metrics = mnist_model.evaluate(X_test, Y_test, verbose=2)\n print(\"Test Loss\", loss_and_metrics[0])\n print(\"Test Accuracy\", loss_and_metrics[1])\n\n #2- test sonrası\n predicted_classes = mnist_model.predict_classes(X_test)\n # see which we predicted correctly and which not\n correct_indices = np.nonzero(predicted_classes == y_test_label)[0]\n incorrect_indices = np.nonzero(predicted_classes != y_test_label)[0]\n print()\n print(len(correct_indices),\" classified correctly\")\n print(len(incorrect_indices),\" classified incorrectly\")\n \ndef draw_9_correct_9_incorrect_predictions(model_name=\"model_history.h5\"): \n # load the model and create predictions on the test set\n mnist_model = load_model(model_name) \n predicted_classes = mnist_model.predict_classes(X_test)\n correct_indices = np.nonzero(predicted_classes == y_test_label)[0]\n incorrect_indices = np.nonzero(predicted_classes != y_test_label)[0]\n # adapt figure size to accomodate 18 subplots\n plt.rcParams['figure.figsize'] = (7,14)\n\n plt.figure()\n # print(\"plot 9 correct predictions\")\n # plot 9 correct predictions\n for i, correct in enumerate(correct_indices[:9]):\n plt.subplot(6,3,i+1)\n plt.imshow(X_test[correct].reshape(28,28), cmap='gray', interpolation='none')\n plt.title(\n \"Predicted: {}, Truth: {}\".format(predicted_classes[correct],\n y_test_label[correct]))\n plt.xticks([])\n plt.yticks([])\n\n #print(\"plot 9 incorrect predictions\")\n # plot 9 incorrect predictions\n for i, incorrect in enumerate(incorrect_indices[:9]):\n plt.subplot(6,3,i+10)\n plt.imshow(X_test[incorrect].reshape(28,28), cmap='gray', interpolation='none')\n plt.title(\n \"Predicted {}, Truth: {}\".format(predicted_classes[incorrect],\n y_test_label[incorrect]))\n plt.xticks([])\n plt.yticks([])\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ceec205cb506dfb9d8e618e3e5d2e6eb4b1fe47"},"cell_type":"markdown","source":"\n# Model - 1 \n## 784-[128-128]-10 Dropout 0.2,Learning Rate==0.001,Activation Function =relu"},{"metadata":{"trusted":true,"_uuid":"3fc902e89128b62b5d3d74fc6375afb3758e05f9","scrolled":false},"cell_type":"code","source":"#parameters\nnum_epochs = 20\nlengh_features = 784 #lengh_features = input layer = 28 pixel * 28 pixel == 784 node\n\n# Create Model1 # 784-128-128-10 Dropout 0.2,Learning Rate==0.001,Activation Function =relu\n# hidden layer start 128 and 2 hidden layer\n# activation relu for popular and fast\nmodel1, history_model1 = create_model(input_length=lengh_features,hidden_lenght1=128,hidden_lenght2=128,\n activation_func='relu',dropout_val=0.20,\n lr_val=0.001,num_epochs=20, batch_size=128\n )\n# saving the model\nsaving_the_model(model1,'model1_history.h5')\n#model scoring\nscoring_the_model('model1_history.h5')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"25f943a8b32177817926283f2db44ee817fa8274"},"cell_type":"code","source":"#Model Drawing to visualize the behavior\ndraw_model(history_model1) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2e6bc4f4a49f7c91b33552c1054ae65beeb8d6af"},"cell_type":"code","source":"#draw_9 correct_9_incorrect_predictions\ndraw_9_correct_9_incorrect_predictions('model1_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8afbac546e5104e41625b849e17c00f80ec0a1c8"},"cell_type":"markdown","source":"\n# Model - 2 \n## 784-[128-128]-10 CHANGE DROPOUT 0.1,Learning Rate==0.001,Activation Function =relu\n## Change Dropout from 0.20 to 0.10"},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"2354553f9c17bf6ef97aa2bc63875c7b648605d1"},"cell_type":"code","source":"# Create Model2 # 784-[128-128]-10 CHANGE DROPOUT 0.1,Learning Rate==0.001,Activation Function =relu\n# Change Dropout from 0.20 to 0.10\nmodel2, history_model2 = create_model(input_length=lengh_features,hidden_lenght1=128,hidden_lenght2=128,\n activation_func='relu',dropout_val=0.10,\n lr_val=0.001,num_epochs=20, batch_size=128)\n# saving the model\nsaving_the_model(model2,'model2_history.h5')\n#model scoring\nscoring_the_model('model2_history.h5') ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2bcde2e9a3b250123ba0e1241c41fa16265444b7"},"cell_type":"code","source":"#Model Drawing to visualize the behavior\ndraw_model(history_model2) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6daefb1c53eb89a16fb0e622c47686cfda0f0b02"},"cell_type":"code","source":"#draw_9 correct_9_incorrect_predictions\ndraw_9_correct_9_incorrect_predictions('model2_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a901de7abd2d8bf0e829c73c1b6ba5b235c5a617"},"cell_type":"markdown","source":"\n# Model - 3 \n## 784-[256-256]-10 Dropout 0.2,Learning Rate==0.001,Activation Function =relu\n## Change Hidden Layer1 Node from 128 to 256\n## Change Hidden Layer1 Node from 128 to 256"},{"metadata":{"trusted":true,"_uuid":"3e25b00270b2ae5bba61495f46dcce009397ac08"},"cell_type":"code","source":"#Change Hidden Layer1 Node from 128 to 256\n#Change Hidden Layer1 Node from 128 to 256\n\n# Create Model3 # 784-[256-256]-10 Dropout 0.2,Learning Rate==0.001,Activation Function =relu\nmodel3, history_model3 = create_model(input_length=lengh_features,hidden_lenght1=256,hidden_lenght2=256,\n activation_func='relu',dropout_val=0.20,\n lr_val=0.001,num_epochs=20, batch_size=128)\n # saving the model\nsaving_the_model(model3,'model3_history.h5')\n#model scoring\nscoring_the_model('model3_history.h5') ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cc9329d2443cdadb169b4dba55b1bc375f6ab649"},"cell_type":"markdown","source":"\n# Compare Model-3 to Model-2\n## Result : 256 Hidden node better than 128 hidden node"},{"metadata":{"trusted":true,"_uuid":"7e255270cd2958bb4eca49d7f4eb284149a5b910"},"cell_type":"code","source":"#Model Drawing to visualize the behavior\ndraw_model(history_model3) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"27d3c9e8df0add57ff13377e28480778be3ec8b3"},"cell_type":"code","source":"#draw_9 correct_9_incorrect_predictions\ndraw_9_correct_9_incorrect_predictions('model3_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"031d9f97deb90098e0b1b6ebba01b6a8b7574a49"},"cell_type":"markdown","source":"\n# Create Model - 4 \n## 784-[256-256]-10 DROPOUT Change from 0.20 to 0.25,Learning Rate==0.001,Activation Function =relu\n## DROPOUT Change from 0.20 to 0.25"},{"metadata":{"trusted":true,"_uuid":"31288f319d7a321cb0fa16562e6833843bae1fe1"},"cell_type":"code","source":"#DROPOUT Change from 0.20 to 0.25\n\n#Create Model4 # 784-[256-256]-10 DROPOUT Change from 0.20 to 0.25,Learning Rate==0.001,Activation Function =relu\nmodel4, history_model4 = create_model(input_length=lengh_features,hidden_lenght1=256,hidden_lenght2=256,\n activation_func='relu',dropout_val=0.25,\n lr_val=0.001,num_epochs=20, batch_size=128)\n # saving the model\nsaving_the_model(model4,'model4_history.h5')\n#model scoring\nscoring_the_model('model4_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bfbab6a07a2bb9527bca5fc38c39d4831eb4f690"},"cell_type":"code","source":"#Model Drawing to visualize the behavior\ndraw_model(history_model4) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1d6555fb5ba80cfc67de296664280f08cadced78"},"cell_type":"code","source":"#draw_9 correct_9_incorrect_predictions\ndraw_9_correct_9_incorrect_predictions('model4_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3da2a407fa0264b21cc018a763c70b25080fd761"},"cell_type":"markdown","source":"\n\n# Create Model - 5\n## 784-[512-512]-10 Dropout 0.25,Learning Rate==0.001,Activation Function =relu\n## Change Hidden Layer-1 Node from 256 to 512\n## Change Hidden Layer-2 Node from 256 to 512"},{"metadata":{"trusted":true,"_uuid":"c60553031a00699d196dc7228172047ba5ae6686"},"cell_type":"code","source":"#Change Hidden Layer1 Node from 256 to 512\n#Change Hidden Layer2 Node from 256 to 512\n\n# Create Model5 # 784-[512-512]-10 Dropout 0.25,Learning Rate==0.001,Activation Function =relu\nmodel5, history_model5 = create_model(input_length=lengh_features,hidden_lenght1=256,hidden_lenght2=256,\n activation_func='relu',dropout_val=0.25,\n lr_val=0.001,num_epochs=20, batch_size=128)\n \n# saving the model\nsaving_the_model(model5,'model5_history.h5')\n#model scoring\nscoring_the_model('model5_history.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"175948001badfacbe75b49c1a93fb2391743573b"},"cell_type":"code","source":"#Model Drawing to visualize the behavior\ndraw_model(history_model5) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"46888acac9673e15e123d233ca9f3ca37a64ed72"},"cell_type":"code","source":"#draw_9 correct_9_incorrect_predictions\ndraw_9_correct_9_incorrect_predictions('model5_history.h5')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}