{"cells":[{"metadata":{},"cell_type":"markdown","source":"The [Assira](https://www.microsoft.com/en-us/research/project/asirra/?from=http%3A%2F%2Fresearch.microsoft.com%2Fen-us%2Fum%2Fredmond%2Fprojects%2Fasirra%2F) Dogs vs. Cats dataset is a great dataset for beginners to get started with image classification. The goal of the prediction task is to generate an output 1 for a Dog image, and 0 for a cat image. \n\nThis notebook contains a detailed EDA on the dataset, followed by some fairly simple Keras CNN architectures. Some model improvement techniques are considered to address overfitting, and we'll evaluate how well these models perform.\n\n"},{"metadata":{"trusted":true,"_uuid":"edf6d792c46851e6296690c570e3d5da6f596d56"},"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nimport numpy as np\nfrom plotly import graph_objs as go\nimport pandas as pd\nimport os\nimport seaborn as sns\nfrom tqdm import tqdm_notebook as tqdm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Table of contents \n1. [Understanding the folder structures](#folders)\n2. [Consolidate labels for images](#labels)\n3. [EDA](#eda)\n    * [Looking at the images](#look)\n    * [Cat vs Dog Frequencies](#freqplot)\n    * [Image dimensions](#dimensions)\n4. [A simple first CNN with Keras](#firstcnn)\n    * [Defining the model architecture](#architecture)\n    * [Data preprocessing](#preprocessing)\n    * [Model evaluation](#evaluationfirst)\n5. [Improving the architecture](#improvedmodel)"},{"metadata":{},"cell_type":"markdown","source":"## <a class='anchor' id ='folders'> Understanding the folder structures\nLet's take a look at the folder structure"},{"metadata":{"trusted":true},"cell_type":"code","source":"os.listdir(\"../input/\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The train data is contained within a .zip file. Lets extract this zip file into a working folder within `../kaggle/working` folder."},{"metadata":{"trusted":true},"cell_type":"code","source":"import zipfile \nwith zipfile.ZipFile(\"../input/\"+\"train\"+\".zip\",\"r\") as z:\n    z.extractall(\"../kaggle/working/temp_unzip\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's take a look at the first ten items in the folder we've extracted:"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"List of first ten image filenames: \\n {os.listdir('../kaggle/working/temp_unzip/train')[:10]}\")\nprint(f\"Total number of images in training data: {len(os.listdir('../kaggle/working/temp_unzip/train'))}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <a class='anchor' id = 'labels'> Consolidate labels for images </a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"filenames = os.listdir('../kaggle/working/temp_unzip/train')\nlabels = [str(x)[:3] for x in filenames]\ntrain_df = pd.DataFrame({'filename': filenames, 'label': labels})\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's encode the categorical labels to 1 or 0"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df['label'] = train_df['label'].map({'dog': '1', 'cat':'0'})\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <a class='anchor' id='eda'> EDA </a>\n\nIn this section I'll explore the dataset in some depth. This includes looking at some examples of each image, exploring the distribution of cat vs dog classes, and studying the image dimensions.\n\n### <a class='anchor' id='looking'> Looking at the images </a>\n\nLe'ts plot 5 instances of each class"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"The data has {train_df['label'].nunique()} unique classes\")\n\nfor lab in train_df['label'].unique(): \n    #Subset to just that target \n    label_df = train_df[train_df['label']==lab].reset_index()\n    cols = 5\n    rows = 1\n    fig = plt.figure(figsize = (4*cols - 1, 4.5*rows - 1))\n    for c in range(cols):\n        for r in range(rows):\n            ax = fig.add_subplot(rows, cols, c*rows + r + 1)\n            img = mpimg.imread('../kaggle/working/temp_unzip/train/'+label_df['filename'][c+r])\n            ax.imshow(img)\n            ax.set_title(str(label_df['filename'][c+r]))\n    fig.suptitle(str(label_df['filename'][c+r][:3].upper()))\n    plt.show()\n    plt.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## <a class='anchor' id='freqplot'> Cat vs Dog frequencies </a>\n\nWe see that the classes are balanced, with 12,500 images in each class. "},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.DataFrame(train_df['label'].value_counts().reset_index())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Image dimensions \n\nLet's break down the image dimensions to understand how they are distributed. This will help us at the modelling stage."},{"metadata":{"trusted":true},"cell_type":"code","source":"dims_dict = {'image': [], 'width': [], 'height': [], 'channels': []}\nfor i in tqdm(range(len(train_df))):#['filename']):#train_pathlabel_df['image'].unique())):\n    dims = mpimg.imread('../kaggle/working/temp_unzip/train/'+train_df['filename'][i]).shape\n    dims_dict['image'].append(train_df['filename'][i])\n    dims_dict['height'].append(dims[0])\n    dims_dict['width'].append(dims[1])\n    dims_dict['channels'].append(dims[2])\n\ndims_df = pd.DataFrame(dims_dict)\ndims_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.distplot(dims_df['height'])\nplt.title('Distribution of image heights');\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.distplot(dims_df['width'])\nplt.title('Distribution of image widths');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets split this between dogs and cats to see if there's any significant difference in the distribution of image dimensions"},{"metadata":{"trusted":true},"cell_type":"code","source":"dims_df['label'] = dims_df['image'].apply(lambda x: x[:3])\ndims_df.head(3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.distplot(dims_df[dims_df['label']=='dog']['height'], label='dog')\nsns.distplot(dims_df[dims_df['label']=='cat']['height'], label='cat')\nplt.title('Distribution of image heights between cats and dogs')\nplt.legend();\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.distplot(dims_df[dims_df['label']=='dog']['width'], label='dog')\nsns.distplot(dims_df[dims_df['label']=='cat']['width'], label='cat')\nplt.title('Distribution of image widths between cats and dogs')\nplt.legend();\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Therefore, the distributions of dimensions between the two classes look similar."},{"metadata":{},"cell_type":"markdown","source":"## <a class='anchor' id='firstcnn'> A simple CNN with Keras </a>\n\n### <a class='anchor' id='architecture'> Model architecture </a>\nLets start with building a simple 2-layer model with Keras. We'll try a fairly simple architecture taken from Francois Chollet's book 'Deep Learning with Python'."},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras import models\nfrom keras import layers\n\nnetwork = models.Sequential()\nnetwork.add(layers.Conv2D(32, (3,3), activation = 'relu', input_shape = (200, 200, 3), name=\"conv_1\"))\nnetwork.add(layers.MaxPooling2D((2,2), name=\"maxpool_1\"))\nnetwork.add(layers.Conv2D(64, (3,3), activation = 'relu', name=\"conv_2\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_2\"))\nnetwork.add(layers.Conv2D(128, (3,3), activation = 'relu', name=\"conv_3\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_3\"))\nnetwork.add(layers.Conv2D(128, (3,3), activation = 'relu', name=\"conv_4\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_4\"))\n\nnetwork.add(layers.Flatten())\nnetwork.add(layers.Dense(512, activation = 'relu', name=\"dense_1\"))\nnetwork.add(layers.Dense(1, activation = 'sigmoid', name=\"dense_2\"))\nnetwork.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"network.compile(optimizer = 'adam',\n               loss = 'binary_crossentropy',\n               metrics = ['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below is a breakdown of each layer and its parameters:\n1. __`Conv2D`__: Takes an input of size `(200, 200, 3)`, passes them through 32 filters (each with size `(3,3)`). Since we haven't specified any padding, the output will be a 3D tensor of shape `(198, 198, 3)`. The number of parameters in this layer will be 896 i.e. `out_channels * (in_channels * kernel_h * kernel_w + 1)` i.e. `32 * (3*3*3+1)` where 1 is for the bias term\n2. __`MaxPooling2D`__: Takes an argument for a window size, and returns an output with a pooled version of the previous layer\n3. __`Flatten`__: Flatten the 3-D output to 1-D\n4. __`Dense`__: Layers with some activation function (e.g. relu). The last `Dense`layer will have a sigmoid activation function to predict the final output. "},{"metadata":{},"cell_type":"markdown","source":"### <a class='anchor' id='preprocessing'> Data preprocessing </a> \n\nWe use the keras `ImageDataGenerator` class to convert the raw images into tensors"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.preprocessing.image import ImageDataGenerator\n# Instantiate an ImageDataGenerator with 30% validation data \ndatagen = ImageDataGenerator(rescale = 1./255,\n                             validation_split=0.3)\n\n# Call the `flow_from_dataframe` method to create a generator for training data. \n# The input dataframe to this method needs to contain the image paths & target labels\ntrain_data_gen = datagen.flow_from_dataframe(dataframe=train_df,\n                                             directory='../kaggle/working/temp_unzip/train/',#Target directory\n                                             x_col = 'filename',\n                                             y_col = 'label',\n                                             class_mode = 'binary',#Since we use binary crossentropy\n                                             target_size=(200, 200),#All images will be resized to this\n                                             color_mode='rgb',\n                                             batch_size = 32,\n                                             shuffle=True,\n                                             seed=42,\n                                             subset = 'training',#Just for train data generation\n                                             validate_filenames=False)\n\n# Repeat for validation data \nval_data_gen  = datagen.flow_from_dataframe(dataframe=train_df,\n                                            directory='../kaggle/working/temp_unzip/train/',\n                                            x_col = 'filename',\n                                            y_col = 'label',\n                                            class_mode = 'binary',\n                                            target_size=(200, 200),#All images will be resized to this\n                                            color_mode='rgb',\n                                            batch_size = 32,\n                                            shuffle=True,\n                                            seed=42,\n                                            subset = 'validation',#Just for train data generation\n#                                             interpolation='bilinear',#Can try nearest as well. Need to read up on this\n                                            validate_filenames=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's examine the outputs of the data generator. The [Keras documentation](https://keras.io/api/preprocessing/image/#flowfromdataframe-method) for `ImageDataGenerator.flow_from_dataframe` says that it: \n\n* Takes the dataframe and the path to a directory + generates batches.\n\n* Returns a DataFrameIterator yielding tuples of (x, y) where x is a numpy array containing a batch of augmented/normalized images with shape (batch_size, *target_size, channels) and y is a numpy array of corresponding labels.\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"for data_array, label_array in train_data_gen:\n    print(f\"Shape of train data batch data is {data_array.shape}\")\n    print(f\"Shape of train data batch labels is {label_array.shape}\")\n    break # The generator has infinite yield, as endlessly iterates over batches. We need to break it manually","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that the train data generator yields 32 batches of RGB images, each of shape 200 * 200 * 3.\n\nThe labels generated are binary labels of shape (20,)."},{"metadata":{},"cell_type":"markdown","source":"### <a class='anchor' id='training'> Training the model </a>"},{"metadata":{},"cell_type":"markdown","source":"The `fit_generator`method is used to train the CNN. Its arguements are:\n* a Python generator that will yield batches of inputs and targets indefinitely\n* `steps_per_epoch`: How many samples to draw from the generator before declaring an epoch over"},{"metadata":{"trusted":true},"cell_type":"code","source":"history = network.fit_generator(train_data_gen,\n                                steps_per_epoch = 100,\n                                epochs=30,\n                                validation_data = val_data_gen,\n                                validation_steps=50\n                               )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's save our model:"},{"metadata":{"trusted":true},"cell_type":"code","source":"network.save('cats_and_dogs_small_model.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### <a class='anchor' id='evaluationfirst'> Model evaluation </a>\n\nLets plot the loss and the accuracy of the model over training and validation data."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(history.history.keys())\ntrain_acc = history.history['acc']\nval_acc = history.history['val_acc']\ntrain_loss = history.history['loss']\nval_loss = history.history['val_loss']\nn_epochs = len(train_acc)\nfig = plt.figure(figsize = (15,8))\nfig.add_subplot(121)\nplt.plot(range(n_epochs), train_acc, color = 'orange', label = \"Train accuracy\")\nplt.plot(range(n_epochs), val_acc, color = 'blue', label = \"Validation accuracy\")\nplt.legend();\nfig.add_subplot(122)\nplt.plot(range(n_epochs), train_loss, color = 'orange', label = \"Train loss\")\nplt.plot(range(n_epochs), val_loss, color = 'blue', label = \"Validation loss\")\nplt.legend();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We observe that: \n* Training accuracy improves with each epoch\n* Validation accuracy on the other hand improves less quickly after around the 26th epoch\n* Training loss similarly decreases consistently\n* Improvement in validation loss tapers off after around the 20th epoch\n\nIn order to prevent overfitting and improve the validation accuracy, lets try a few model improvements.\n\n## <a class='anchor' id='improvedmodel'> Improving the architecture </a>\n\nLets run the below data augmentation steps on the training data: \n* Rotate images within a range of 45 degrees\n* Horizontal and vertical translation of images \n* Zoom into some images randomly \n* Apply shear transformations"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.preprocessing.image import ImageDataGenerator\n\n# ImageDataGenerator for validation data \ntrain_datagen = ImageDataGenerator(rescale = 1./255,\n                                   rotation_range = 45,\n                                   width_shift_range = 0.2,\n                                   height_shift_range = 0.2,\n                                   shear_range = 0.2,\n                                   zoom_range = 0.2)\nval_datagen = ImageDataGenerator(rescale = 1./255)\n\n# Call the `flow_from_dataframe` method to create a generator for training data. \n# The input dataframe to this method needs to contain the image paths & target labels\n\n# We split the training and validation data because augmentations/transformations should not be applied to valdation data \nfrom sklearn.model_selection import train_test_split\naugmented_mod_train_df, augmented_mod_val_df = train_test_split(train_df, test_size=0.2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets take a quick look at class balances in the train and validation data "},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"Split between classes in train data: \\n {augmented_mod_train_df['label'].value_counts()*100 / augmented_mod_train_df.shape[0]}\")\nprint(f\"Split between classes in validation data: \\n {augmented_mod_val_df['label'].value_counts()*100 / augmented_mod_val_df.shape[0]}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The distribution looks very similar to the original distribution in the training data. \nLet's run the data augmentation pipeline and train the network."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data_gen = train_datagen.flow_from_dataframe(dataframe=augmented_mod_train_df,\n                                             directory='../kaggle/working/temp_unzip/train/',#Target directory\n                                             x_col = 'filename',\n                                             y_col = 'label',\n                                             class_mode = 'binary',#Since we use binary crossentropy\n                                             target_size=(200, 200),#All images will be resized to this\n                                             color_mode='rgb',\n                                             batch_size = 32,\n                                             shuffle=True,\n                                             seed=42,\n                                             validate_filenames=False)\n\n# Repeat for validation data \nval_data_gen  = val_datagen.flow_from_dataframe(dataframe=augmented_mod_train_df,\n                                            directory='../kaggle/working/temp_unzip/train/',\n                                            x_col = 'filename',\n                                            y_col = 'label',\n                                            class_mode = 'binary',\n                                            target_size=(200, 200),#All images will be resized to this\n                                            color_mode='rgb',\n                                            batch_size = 32,\n                                            shuffle=True,\n                                            seed=42,\n                                            validate_filenames=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's add a dropout layer before our final classification step, and then retrain our earlier network."},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras import models\nfrom keras import layers\nfrom keras import optimizers\n\nnetwork = models.Sequential()\nnetwork.add(layers.Conv2D(32, (3,3), activation = 'relu', input_shape = (200, 200, 3), name=\"conv_1\"))\nnetwork.add(layers.MaxPooling2D((2,2), name=\"maxpool_1\"))\nnetwork.add(layers.Conv2D(64, (3,3), activation = 'relu', name=\"conv_2\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_2\"))\nnetwork.add(layers.Conv2D(128, (3,3), activation = 'relu', name=\"conv_3\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_3\"))\nnetwork.add(layers.Conv2D(128, (3,3), activation = 'relu', name=\"conv_4\"))\nnetwork.add(layers.MaxPooling2D((2,2), name = \"maxpool_4\"))\n\nnetwork.add(layers.Flatten())\nnetwork.add(layers.Dropout(0.2))\nnetwork.add(layers.Dense(512, activation = 'relu', name=\"dense_1\"))\nnetwork.add(layers.Dense(1, activation = 'sigmoid', name=\"dense_2\"))\nnetwork.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"network.compile(optimizer = optimizers.adam(lr=1e-4),\n               loss = 'binary_crossentropy',\n               metrics = ['accuracy'])\n\nhistory = network.fit_generator(train_data_gen,\n                                steps_per_epoch = 50,\n                                epochs=50,\n                                validation_data = val_data_gen,\n                                validation_steps=50\n                               )\n# network.save('cats_and_dogs_augmented_data.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_acc = history.history['acc']\nval_acc = history.history['val_acc']\ntrain_loss = history.history['loss']\nval_loss = history.history['val_loss']\nn_epochs = len(train_acc)\nfig = plt.figure(figsize = (15,8))\nfig.add_subplot(121)\nplt.plot(range(n_epochs), train_acc, color = 'orange', label = \"Train accuracy\")\nplt.plot(range(n_epochs), val_acc, color = 'blue', label = \"Validation accuracy\")\nplt.legend();\nfig.add_subplot(122)\nplt.plot(range(n_epochs), train_loss, color = 'orange', label = \"Train loss\")\nplt.plot(range(n_epochs), val_loss, color = 'blue', label = \"Validation loss\")\nplt.legend();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's evalute the performance of the model on the test data"},{"metadata":{"trusted":true},"cell_type":"code","source":"import zipfile \nwith zipfile.ZipFile(\"../input/\"+\"test1\"+\".zip\",\"r\") as z:\n    z.extractall(\"../kaggle/working/temp_test_unzip\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"filenames = os.listdir('../kaggle/working/temp_test_unzip/test1')\ntest_df = pd.DataFrame({'filename': filenames})\nprint(test_df.shape)\ntest_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's create a test data generator in the same way that we created a train & validation data generator"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data_gen  = val_datagen.flow_from_dataframe(dataframe=test_df,\n                                            directory='../kaggle/working/temp_test_unzip/test1/',\n                                            x_col = 'filename',\n                                            y_col = None,\n                                            class_mode = None,\n                                            target_size=(200, 200),#All images will be resized to this\n                                            color_mode='rgb',\n                                            batch_size = 64,\n                                            shuffle=False)#,\n#                                             seed=42,\n#                                             validate_filenames=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"yhat = network.predict_generator(test_data_gen, steps=np.ceil(test_df.shape[0]/64))\nprint(yhat.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_df = test_df.copy()\nsubmission_df['id'] = submission_df['filename'].str.split('.').str[0]\nsubmission_df['label'] = yhat\nsubmission_df['label'] = np.where(yhat>0.5, 1, 0)\nsubmission_df[['id', 'label']].to_csv('submission.csv', index=False)\nsubmission_df[['id', 'label']].head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's it for a CNN architecture implementation in Keras, in my next notebook I'll attempt transfer learning on the dataset to improve the accuracy. "},{"metadata":{},"cell_type":"markdown","source":"References: \n\n1. [Deep Learning with Python, Francois Chollet](https://github.com/sri-spirited/fchollet-book-deep-learning-with-python-notebooks)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}