{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Building a Basic RESNET Based Transfer Learning Network: The MNIST Dataset\nIn this Notebook i present a basic rundown of using the RESNET50 achitecture for transfer learning \n10 digits (0-9) of the MNIST dataset. The objective of this notebook is to build a sample transfer network uing Keras.\n\n* Importing libraries and the dataset\n* Data preparation: Train-test split, specifying the shape of the input data etc.\n* Building and understanding the Transfer Learning process\n* Fitting and evaluating the model\n\n \n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"## Import relevant libraries\n\nimport numpy as np\nimport random\nimport keras\nfrom keras.datasets import mnist\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D\nfrom keras import backend as K\n\n\nimport matplotlib.pyplot as plt\n\nfrom datetime import datetime\nnow = datetime.now()\ncurrent_time = now.strftime(\"%H:%M:%S\")\nprint(current_time)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## load the dataset into the frame\ndf_train = pd.read_csv('../input/digit-recognizer/train.csv')\ndf_test = pd.read_csv('../input/digit-recognizer/test.csv')\n\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## we remove the label from train data  in seperate columns and data in seperate test data has no labels \n## convert the pd dataframe to numpy arrays for processing  \ny_data = df_train.label.to_numpy()\nx_data = df_train.drop('label',axis=1).to_numpy()\n\nprint(x_data.shape,y_data.shape)\n\nx_test = df_test.to_numpy()\nprint(x_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"##   now we reshape the data first to 28x28 view for image processing since we will be using images processing routine\n##   normalization i done by dividing by 255 as simple way : \n\n\nx_train = x_data.reshape(x_data.shape[0],28,28)\nx_test = x_test.reshape(x_test.shape[0],28,28)\n\n\n# convert int to float for all data and normalize using divide by 255.0 : \nx_train = x_train.astype('float32')\nx_test = x_test.astype('float32')\n\n# normalise\nx_train /= 255.0\nx_test /= 255.0\n\nprint(x_train.shape,x_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"FOr processing in Resnet teh architecture expectes the input to be in 4D array \n* shape size : recordsxwidthxheightxchannals in RGB/RGBA or 3,4 channal format \n* all our images here are greaycale i.e 1 channal format , \n* we will convert the 1 cannal greyscale images to 3 channel RGB images for input feed\n* Input shape will be BATCH x IMG x IMG x 1\n* Output Shape will be BATCH x IMG x IMG x 3"},{"metadata":{"trusted":true},"cell_type":"code","source":"## we convert our train,test data to 4D format , by repeating the image channal on 3 axis\n## as per belw code, so we copy same array 28x28 in a 3D matrix 3 times across axis z \n## using repeat() function from numpy which is a very underrated function \n\nrgb_train_batch = np.repeat(x_train[..., np.newaxis], 3, -1)  ## repeat array across convert multiple axis \nrgb_test_batch  = np.repeat(x_test[..., np.newaxis], 3, -1)  ## repeat array across convert multiple axis \n\nprint(rgb_train_batch.shape,rgb_test_batch.shape)\n\n## the output will be a matrix in shape ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"## lets test a random image to see if its still visible \nplt.imshow(rgb_train_batch[1],cmap=\"gray\")\nplt.imshow(rgb_test_batch[1],cmap=\"gray\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### RESNET 50 Api Resizing Images \n* Resnet has a minimum image size of 32x32x3\n* since our images are not correct size we will resize all our images using Keras and \n* OpenCV to the correct Size from 28x28x3 to 32x32x3 \n* you have to resize this if any image you are feeding to network is not in correct shape, you can get correct shape from the keras Resnet Api "},{"metadata":{"trusted":true},"cell_type":"code","source":"## we set the minimum mage size as 32x32X3 this is minimum image size accepted by keras for resnet , \n## since our image is leass than this we will resize all to 32x32 in feeding\nimport cv2 \n\nIMG_HEIGHT = 32\nIMG_WIDTH = 32\nCHANNAL = 3 \n\ninput_shape=(IMG_HEIGHT,IMG_WIDTH)  ## input shape \n\n## create a dummy array o hold 4D data n correct shape \nresized_train = np.ndarray(shape=(rgb_train_batch.shape[0],IMG_HEIGHT,IMG_WIDTH,CHANNAL))\nresized_test = np.ndarray(shape=(rgb_test_batch.shape[0],IMG_HEIGHT,IMG_WIDTH,CHANNAL))\n\n\n## loop to resize all imags in array and copy to dummy variable created above \nfor rec in range(rgb_train_batch.shape[0]):\n    resized_train[rec] = cv2.resize(rgb_train_batch[rec], (IMG_HEIGHT,IMG_WIDTH),interpolation = cv2.INTER_AREA)\n    \nfor rec in range(rgb_test_batch.shape[0]):\n    resized_test[rec] = cv2.resize(rgb_test_batch[rec], (IMG_HEIGHT,IMG_WIDTH) ,interpolation = cv2.INTER_AREA )\n\n\nprint(resized_train.shape, resized_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"## NOw the images are resized , check a sampel image to see if data is there \nplt.imshow(resized_test[1],cmap=\"gray\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## we do a train test split and the test data here is be used for validation set , the actual test data is already decided and loaded in x_test\nif you dont want to manucally do it , since data is less we can always use the **validation_split** parameter in fit function wil work as well."},{"metadata":{"trusted":true},"cell_type":"code","source":"## we do a train test split \nfrom sklearn.model_selection import train_test_split\n\nx_train, val_train, y_train, val_test= train_test_split(resized_train,y_data,test_size=0.2,train_size=0.8,shuffle=True)\nprint(x_train.shape,y_train.shape, val_train.shape ,val_test.shape)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### here we start with importing and setting up resnet50 model and  for data processing\n* we import the resnet50 model from keras API\n* The model weights are set to imagenet, which is a 1000 classes pretrained \n* The Top layer is removed , we will introduce our out output layers on top of it for final output\n* The Trainable layers are set to False, this isdone so only the new layers are trained, all RESNET layer  are not trained\n* we can retrain the network on our dataset which will give better results, bt that defeates the point of transfer learning , a modified approach is to disable learning for few seleted layers\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"### sart with importing and setting up resnet50 model for data processing\n\nfrom keras.applications.resnet50 import ResNet50\nfrom keras.models import Model\nfrom keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, InputLayer\nfrom keras.models import Sequential\nfrom keras import optimizers\ninput_shape=(IMG_HEIGHT,IMG_WIDTH,CHANNAL) ## we have to explicitely tell keras the input size\n\n\n\n\npretrained_model = keras.applications.resnet50.ResNet50(input_shape=input_shape, \n                                                        include_top=False, weights= 'imagenet' )\n\n# set model weights as non trainable  \n## the model can be again tested with partially trainable paramss you can use below code as needed\nfor l in pretrained_model.layers:\n        l.trainable = False\n   \n## -----\n\nmodel = keras.Sequential([\n    pretrained_model,\n    keras.layers.Flatten(),\n    keras.layers.Dense(units=512, activation='relu'),\n    keras.layers.Dense(units=128, activation='relu'),\n    keras.layers.Dense(units=64, activation='relu'),\n    keras.layers.Dense(units=10, activation='softmax')\n])\n\n\nmodel.summary()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## we chose SparseCategoricalCrossentropy as loss since our pretrained model has 1000 classes and our current model has 10 classes \n* this gives erros in case of using CategoricalCrossentropy or other lossess\n* if we retrain the entire model weights using **layers.trainable=True** as previous we can use other losses \n* Retraining entire Network with nw weights will provide better accuracy keeping other things same "},{"metadata":{"trusted":true},"cell_type":"code","source":"## here we set and compile the model \n## we chose SparseCategoricalCrossentropy as loss since our pretrained model has 1000 classes and our current model\n## has 10 classes , this gives erros in case of using CategoricalCrossentropy or other lossess\n## if we retrain the entire model weights using layers.trainable=True as precious we can use other losses also \n## prviding better accuracy keeping other things same \n\nOptimizer = keras.optimizers.Adam() \nlosses = keras.losses.SparseCategoricalCrossentropy()\n\nmodel.compile(optimizer=Optimizer, \n                loss= losses ,\n                metrics=['accuracy'])\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"### execute the model fit \n\nbatch_size = 128\nepochs = 40 \n\nhistory = model.fit(x_train, y_train,\n                    batch_size=batch_size,\n                    epochs=epochs,\n                    verbose=1,\n                    validation_data=(val_train, val_test))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## plot the loss and accuracy metrics "},{"metadata":{"trusted":true},"cell_type":"code","source":" \ntrain_acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']  \ntrain_loss = history.history['loss'] \nval_loss = history.history['val_loss'] \n\nepochs = range(epochs)\n\n# subplots \nfig, ax = plt.subplots(1, 2, figsize=(15, 5))\nfig.subplots_adjust(wspace=0.15, hspace=0.025)\nax = ax.ravel() ## plot arrary \n\n\n\nax[0].plot(epochs, train_acc, 'r', label='Training accuracy')\nax[0].plot(epochs, val_acc, 'b', label='Validation accuracy')\nax[0].set_title('Training and validation accuracy')\nax[0].legend()\n\n\nax[1].plot(epochs, train_loss, 'r', label='Training Loss')\nax[1].plot(epochs, val_loss, 'b', label='Validation Loss')\nax[1].set_title('Training and validation loss')\nax[1].legend()\n\nplt.show()\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### we evaluate and predict our output model data\nThe output predictions will be a prob distribution in 10 classes , extracting final results from them ,we create the dataset for submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"##predicton \n\n#y_Pred = model.predict(resized_test,steps = resized_test.shape[0], verbose = 1) \ny_Pred = model.predict(resized_test,verbose = 1)\n## the final predicted labels are stored here \npredicted_class = np.argmax(y_Pred, axis = 1) ## i changed steps to show the verbose outputs, select max prob from array index\npredicted_class.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"##lets checkthe predicted outputs \n## lets test 1 image and lable of 0th position : looks good \nplt.imshow(x_test[0].reshape(28,28), cmap=plt.get_cmap('gray'), vmin=0, vmax=1)\n# print the predicted label of the above image\npredicted_class[0]\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## create the submission file with predicted outputs"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pandas as pd \n## looks good \n# create csv ataset for redictions and save for submission\n\nsubmissions = pd.DataFrame()\nsubmissions[\"ImageId\"] = [i for i in range(1, predicted_class.shape[0]+1)]\nsubmissions[\"Label\"] = predicted_class\n\nsubmissions.to_csv(\"submissions.csv\", index=False)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}