{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\nThis kernel will explore image processing using a convolutional neural network running on tensor backend with keras as the programming interface.  The Dogs vs. Cats dataset will be used in this study, data augmentation will be applied to both a custom CNN and VGG16 feature extraction with fine tuning."},{"metadata":{},"cell_type":"markdown","source":"## 1.0 Data exploration\nLoad and explore data"},{"metadata":{},"cell_type":"markdown","source":"The dataset for this kernel will be the Dogs vs. Cats dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"#load libraries for data manipulation and visualization\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sb\nimport os\nimport random\nfrom tensorflow.keras.preprocessing.image import load_img\n# warnings\nimport string\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Create data directories"},{"metadata":{"trusted":true},"cell_type":"code","source":"!unzip -q /kaggle/input/dogs-vs-cats/train.zip\n!unzip -q /kaggle/input/dogs-vs-cats/test1.zip\nTRAIN_DIR = \"/kaggle/working/train/\"\nTEST_DIR = \"/kaggle/working/test1/\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# gather train data into a dataframe\nfilenames = os.listdir(TRAIN_DIR )\ncategories = []\nfor filename in filenames:\n    category = filename.split('.')[0]\n    if category == 'dog':\n        categories.append('dog')\n    else:\n        categories.append('cat')\n\nall_df = pd.DataFrame({\n    'filename': filenames,\n    'category': categories\n})\n\n# gather test data into a dataframe\ntest_filenames = os.listdir(TEST_DIR)\ntest_df = pd.DataFrame({\n   'id': test_filenames\n})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# display train data\nall_df.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# show counts for train data\nall_df['category'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# display test data\ntest_df.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# show sample size of test data\ntest_df.shape[0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Visualizing train data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# display sample train images\nsample = all_df.head(9)\nsample.head()\nplt.figure(figsize=(12, 12))\nfor index, row in sample.iterrows():\n    filename = row['filename']\n    category = row['category']\n    img = load_img(TRAIN_DIR+filename, target_size=(96,96))\n    plt.subplot(3, 3, index+1)\n    plt.imshow(img)\n    plt.xlabel(filename)\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.0  Data Preparation\nSplit the whole dataset into training, validation and test sets"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n# split into train/validate \ntrain_df, validate_df = train_test_split(all_df, test_size=0.20, random_state=0)\ntrain_df = train_df.reset_index(drop=True)\nvalidate_df = validate_df.reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The train set has 20,000 samples while the validate set has 5,000 samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"# show the count by category for train set\ntrain_df['category'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# show the count by category for validate set\nvalidate_df['category'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3.0 Data preprocessing\nData augmentation and transformation of  jpeg image files on disk to floating point tensors"},{"metadata":{},"cell_type":"markdown","source":"Data augmentation increases the sample size by slightly altering the given sample to generate more samples. In a CNN the larger the training sample the better, a large sample enables a CNN to read/extract features from sample without the need for feature engineering. In cases where the sample size is small, the performance of a CNN can be improved by data augmentation."},{"metadata":{},"cell_type":"markdown","source":"The data exists on disk as jpeg image files, the files need to be decoded to RGB grids of pixels and converted to floating point tensors. The pixel values lie between (0, 255) and will be rescaled to (0, 1) interval for faster processing. The Keras class ImageDataGenerator can automatically turn image files on disk to preprocessed tensors will be utilized for the transformation."},{"metadata":{"trusted":true},"cell_type":"code","source":"# define train data augmentation configuration\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\ntrain_datagen = ImageDataGenerator(\n        rescale=1./255,\nrotation_range=30,\nwidth_shift_range=0.15,\nheight_shift_range=0.15,\nshear_range=0.15,\nzoom_range=0.15,\nhorizontal_flip=True,\nfill_mode='nearest')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Generating sample images to illustrate augmentation"},{"metadata":{"trusted":true},"cell_type":"code","source":"# using ImageDataGenerator to generate sample images\nsample_df = train_df.sample(n=1).reset_index(drop=True)\nsample_generator = train_datagen.flow_from_dataframe(\n    sample_df, \n    TRAIN_DIR, \n    x_col='filename',\n    y_col='category',\n    target_size = (128, 128),\n    class_mode='categorical'\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(8, 8))\nfor i in range(0, 4):\n    plt.subplot(2, 2, i+1)\n    for X, Y in sample_generator:\n        image = X[0]\n        plt.imshow(image)\n        break\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Reading train, validation and test data from disk and converting to floating point tensors using ImageDataGenerator."},{"metadata":{"trusted":true},"cell_type":"code","source":"# reading train data\ntrain_generator = train_datagen.flow_from_dataframe(\n        train_df, \n        TRAIN_DIR,\n        x_col='filename',\n        y_col='category',\n        target_size=(128, 128),\n        batch_size=75,\n        class_mode='binary')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# reading validation data\ntest_datagen = ImageDataGenerator(rescale=1./255)\nvalidation_generator = test_datagen.flow_from_dataframe(\n        validate_df, \n        TRAIN_DIR,\n        x_col='filename',\n        y_col='category',\n        target_size=(128, 128),\n        batch_size=50,\n        class_mode='binary')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# reading test data\ntest_generator = test_datagen.flow_from_dataframe(\n        test_df, \n        TEST_DIR,\n        x_col='id',\n        y_col=None,\n        class_mode=None,\n        target_size=(128, 128),\n        batch_size=12500//50)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4.0 Pretrained CNN\nA pretrained CNN can be used to extract features from a small sample if it contains related information with improvement in performance. The Pretrained CNN for this study is the vGG16  trained on ImageNet dataset with many classes including different breeds of cats and dogs. There are 2 steps to using a pretrained CNN: feature extraction and fine tuning."},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"# using the pretrained convolutional base\nfrom tensorflow.keras.applications import VGG16\nconv_base = VGG16(weights='imagenet',\ninclude_top=False,\ninput_shape=(128, 128, 3))\nconv_base.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Feature extraction will be performed by adding dense layers on top of the conv-base and running it end to end on the input data.The feature map from the conv-base that will be passed to the densely connected classifier has a shape of (4,4,512)."},{"metadata":{"trusted":true},"cell_type":"code","source":"from tensorflow.keras import layers\nfrom tensorflow.keras import models\nmodel = models.Sequential()\nmodel.add(conv_base)\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(256, activation='relu'))\nmodel.add(layers.Dense(1, activation='sigmoid'))\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Feature extraction is freezing the conv-base before compilation, freezing prevents the weights of a layer from being updated during training, only the weights of the classifier will be updated during training. Fine tuning consists of unfreezing a few of the top layers frozen during feature extraction and training the newly added layer and the fully connected layer.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# freezing all layers up to a specific one\nconv_base.trainable = True\nset_trainable = False\nfor layer in conv_base.layers:\n    if layer.name == 'block5_conv1':\n        set_trainable = True\n    if set_trainable:\n        layer.trainable = True\n    else:\n        layer.trainable = False\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# optimizing model performance\nfrom tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau\ncallbacks = [\n    EarlyStopping(patience=5, verbose=1),\n    ReduceLROnPlateau(factor=0.1, patience=2, min_lr=0.000001, verbose=1),\n    ModelCheckpoint('model3.h5', verbose=1, save_best_only=True, save_weights_only=True)\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# configuring the model for training\nimport  tensorflow.keras.optimizers as optimizers\nmodel.compile(optimizer=optimizers.RMSprop(lr=1e-5),\nloss='binary_crossentropy',\nmetrics=['acc'])\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fitting the model \nhistory = model.fit_generator(\n        train_generator,\n        steps_per_epoch=200,\n        epochs=30,\n        validation_data=validation_generator,\n        validation_steps=100,\n        callbacks=callbacks)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plotting the results\nimport matplotlib.pyplot as plt\nacc = history.history['acc']\nval_acc = history.history['val_acc']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'r', label='Training acc')\nplt.plot(epochs, val_acc, 'b', label='Validation acc')\nplt.title('Training and validation accuracy')\nplt.legend()\nplt.figure()\nplt.plot(epochs, loss, 'r', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Evaluating Pretrained CNN"},{"metadata":{"trusted":true},"cell_type":"code","source":"# making prediction\npredictions2 = model.predict_generator(test_generator, steps=np.ceil(12500/50))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# converting predictions to 1 and 0\npredictions2 = [1 if y > 0.5 else 0 for y in predictions2]\n\ntest_df['label'] = predictions2\n\n# restore back to class names (dog or cat)\nlabel_map = dict((v,k) for k,v in train_generator.class_indices.items())\ntest_df['label'] = test_df['label'].replace(label_map)\n\n# encoding according to submission format, dog = 1, cat = 0\ntest_df['label'] = test_df['label'].replace({ 'dog': 1, 'cat': 0 })\n\ntest_df.to_csv('submission2.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submit_df.sample(5)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}