{"nbformat": 4, "metadata": {"language_info": {"codemirror_mode": {"version": 3, "name": "ipython"}, "file_extension": ".py", "mimetype": "text/x-python", "pygments_lexer": "ipython3", "nbconvert_exporter": "python", "version": "3.6.3", "name": "python"}, "_is_fork": false, "kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}, "_change_revision": 0}, "cells": [{"cell_type": "markdown", "source": ["Basic Convolutional Neural Network (Convnet) implemented in Keras to classify pictures of cats and dogs.\n", "\n", "Inspiration for this notebook comes from this [Keras blog post](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html) and the [VGG ConvNet paper](https://arxiv.org/pdf/1409.1556.pdf), notebook shamefully stolen from [Jeff Delaney](https://www.kaggle.com/jeffd23). \n"], "metadata": {"_uuid": "3e966a757d5d329ed72313ff6fafeec43d45d0fe", "_cell_guid": "764fe135-c09e-9769-5794-500867154d93"}}, {"cell_type": "code", "execution_count": null, "source": ["import os, cv2, random\n", "import numpy as np\n", "import pandas as pd\n", "\n", "import matplotlib.pyplot as plt\n", "from matplotlib import ticker\n", "import seaborn as sns\n", "%matplotlib inline \n", "\n", "from keras import backend as K\n", "from keras.models import Sequential\n", "from keras.layers import Input, Dropout, Flatten, Conv2D, MaxPooling2D, Dense, Activation\n", "from keras.optimizers import RMSprop\n", "from keras.callbacks import ModelCheckpoint, Callback, EarlyStopping\n", "from keras.utils import np_utils"], "metadata": {"_uuid": "d8ca27de093971c0fbff28d6ffc943b62f3090a7", "_cell_guid": "3d458c15-e131-f3c4-f756-843d6454bb37"}, "outputs": []}, {"cell_type": "markdown", "source": ["## Preparing the Data\n", "\n", "This function resizes the images to 256x256 and samples 20000 images of the data to run efficiently as a Kaggle Kernel. I also separated kitties and puppies for exploratory analysis. "], "metadata": {"_uuid": "3089b960097a7efcdba881fb9bccb3331091691d", "_cell_guid": "04681981-55ac-7820-a0ae-38f98c851c39"}}, {"cell_type": "code", "execution_count": null, "source": ["import os, cv2, random\n", "import numpy as np\n", "import pandas as pd\n", "\n", "TRAIN_DIR = '../input/train/'\n", "TEST_DIR = '../input/test/'\n", "\n", "ROWS = 256\n", "COLS = 256\n", "ROWS2 = 64\n", "COLS2 = 64\n", "CHANNELS = 3\n", "\n", "train_images = [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR)] # use this for full dataset\n", "train_dogs =   [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR) if 'dog' in i]\n", "train_cats =   [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR) if 'cat' in i]\n", "\n", "test_images =  [TEST_DIR+i for i in os.listdir(TEST_DIR)]\n", "\n", "\n", "# slice datasets for memory efficiency on Kaggle Kernels, delete if using full dataset\n", "train_images = train_dogs[:10000] + train_cats[:10000]\n", "random.shuffle(train_images)\n", "test_images =  test_images[:1000]\n", "\n", "def read_image(file_path):\n", "    img = cv2.imread(file_path, cv2.IMREAD_COLOR) #cv2.IMREAD_GRAYSCALE\n", "    b,g,r = cv2.split(img)\n", "    img2 = cv2.merge([r,g,b])\n", "    return cv2.resize(img2, (ROWS2, COLS2), interpolation=cv2.INTER_CUBIC)\n", "\n", "def read_image2(file_path):\n", "    img = cv2.imread(file_path, cv2.IMREAD_COLOR) #cv2.IMREAD_GRAYSCALE\n", "    b,g,r = cv2.split(img)\n", "    img2 = cv2.merge([r,g,b])\n", "    return cv2.resize(img2, (ROWS, COLS), interpolation=cv2.INTER_CUBIC)\n", "\n", "\n", "def prep_data(images):\n", "    count = len(images)\n", "    data = np.ndarray((count, CHANNELS, ROWS2, COLS2), dtype=np.uint8)\n", "\n", "    for i, image_file in enumerate(images):\n", "        image = read_image(image_file)\n", "        data[i] = image.T\n", "        if i%1000 == 0: print('Processed {} of {}'.format(i, count))\n", "    \n", "    return data\n", "\n", "def prep_data2(images):\n", "    count = len(images)\n", "    data = np.ndarray((count, CHANNELS, ROWS, COLS), dtype=np.uint8)\n", "\n", "    for i, image_file in enumerate(images):\n", "        image = read_image2(image_file)\n", "        data[i] = image.T\n", "        if i%500 == 0: print('Processed {} of {}'.format(i, count))\n", "    \n", "    return data\n", "\n", "train = prep_data(train_images)\n", "test = prep_data(test_images)\n", "test2 = prep_data2(test_images)\n", "\n", "print(\"Train shape: {}\".format(train.shape))\n", "print(\"Test shape: {}\".format(test.shape))"], "metadata": {"_uuid": "26e102681b77e889856b6b3fc76ef6b0b4975f1e", "_cell_guid": "663d335e-1b84-a8cb-19ee-8f04839cf4e5"}, "outputs": []}, {"cell_type": "markdown", "source": ["### Generating the Labels\n", "\n", "We're dealing with a binary classification problem here - (1) dog (0) cat. The lables can be created by looping over the file names in the train directory. The graph shows us that the training data has an equal number of cats and dogs."], "metadata": {"_uuid": "ba3a9913ae07ef3f0e83ad92a9cca80dd54dd2f1", "_cell_guid": "ed0fc95a-53bc-3517-245d-cf1f5122bc2d"}}, {"cell_type": "code", "execution_count": null, "source": ["labels = []\n", "for i in train_images:\n", "    if 'dog' in i:\n", "        labels.append(1)\n", "    else:\n", "        labels.append(0)\n", "\n", "sns.countplot(labels)"], "metadata": {"_uuid": "acfe6eec00a262460a1805f2f72c811a097a920b", "_cell_guid": "c0f2fbf6-8d78-7d42-6579-5486a36c1e60"}, "outputs": []}, {"cell_type": "markdown", "source": ["### Checking out the Cats and Dogs\n", "The best part: what do these catts & dogs actually look like? **hint: this is the best part!**"], "metadata": {"_uuid": "1235c0ba9cd6b66b757be4ff5fc0d6643ac52108", "_cell_guid": "4bb0bd2a-0512-aa72-c628-5b6ac946d97c"}}, {"cell_type": "code", "execution_count": null, "source": ["def show_cats_and_dogs(idx):\n", "    cat = read_image2(train_cats[idx])\n", "    dog = read_image2(train_dogs[idx])\n", "    pair = np.concatenate((cat, dog), axis=1)\n", "    plt.figure(figsize=(10,5))\n", "    plt.imshow(pair)\n", "    plt.show()\n", "    \n", "for idx in range(0,5):\n", "    show_cats_and_dogs(idx)"], "metadata": {"_uuid": "70e4c96c514a6de1d5d8c4c9239e0bfe71f08f9a", "_cell_guid": "0b7c79ae-6543-2ed8-e3f5-af4f5beb9cf7"}, "outputs": []}, {"cell_type": "markdown", "source": ["## CatdogNet-16\n", "\n", "A scaled down version of the VGG-16, with a few notable changes.\n", "\n", "- Number of convolution filters cut in half, fully connected (dense) layers scaled down. \n", "- Optimizer changed to `RMSprop`. \n", "- Output layer activation set to `sigmoid` for binary crossentropy. \n", "- Some layers commented out for efficiency.\n", "\n", "The full network takes about 80s per epoch on a GTX1070 (or 2hr+ on CPU) on the full dataset.  (This script only trains on 8% of the 25K images. )"], "metadata": {"_uuid": "99b588b74e44cc77d85c5f144c56b43f3805cdf7", "_cell_guid": "51e403b6-bcfc-b4fa-3770-9850cc86bae3"}}, {"cell_type": "code", "execution_count": null, "source": ["from keras.models import Sequential\n", "from keras.layers import Input, Dropout, Flatten, Conv2D, MaxPooling2D, Dense, Activation\n", "from keras.optimizers import RMSprop\n", "from keras.callbacks import ModelCheckpoint, Callback, EarlyStopping\n", "from keras.utils import np_utils\n", "\n", "optimizer = RMSprop(lr=1e-4)\n", "objective = 'binary_crossentropy'\n", "\n", "\n", "def catdog():\n", "    \n", "    model = Sequential()\n", "\n", "    model.add(Conv2D(32, 3, padding='same', input_shape=train.shape[1:], activation='relu'))\n", "    model.add(Conv2D(32, 3, padding='same', activation='relu'))\n", "    model.add(MaxPooling2D(pool_size=(2, 2), data_format=\"channels_first\"))\n", "    #print(\"First layer...\")\n", "    model.add(Conv2D(64, 3, padding='same', activation='relu'))\n", "    model.add(Conv2D(64, 3, padding='same', activation='relu'))\n", "    model.add(MaxPooling2D(pool_size=(2, 2), data_format=\"channels_first\"))\n", "    #print(\"Second layer...\")\n", "    model.add(Conv2D(128, 3, padding='same', activation='relu'))\n", "    model.add(Conv2D(128, 3, padding='same', activation='relu'))\n", "    model.add(MaxPooling2D(pool_size=(2, 2), data_format=\"channels_first\"))\n", "    #print(\"Third layer...\")\n", "    model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    model.add(MaxPooling2D(pool_size=(2, 2), data_format=\"channels_first\"))\n", "\n", "    #model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    #model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    #model.add(Conv2D(256, (3, 3), padding='same', activation='relu'))\n", "    #model.add(MaxPooling2D(pool_size=(2, 2)))\n", "    #print(\"Flattening, etc...\")\n", "    model.add(Flatten())\n", "    model.add(Dense(256, activation='relu'))\n", "    model.add(Dropout(0.5))\n", "    \n", "    model.add(Dense(256, activation='relu'))\n", "    model.add(Dropout(0.5))\n", "\n", "    model.add(Dense(1))\n", "    model.add(Activation('sigmoid'))\n", "    print(\"Compiling model...\")\n", "    model.compile(loss=objective, optimizer=optimizer, metrics=['accuracy'])\n", "    return model\n", "\n", "print(\"Creating model:\")\n", "model = catdog()"], "metadata": {"_uuid": "5c90f242a2b87d90719a2bd87a6b5b9642c8f960", "_cell_guid": "4c477612-41d3-3112-5176-3c4ad9633080"}, "outputs": []}, {"cell_type": "markdown", "source": ["### Train and Predict\n", "\n", "Here we train the model until the validation loss stops improving, so the model doesn't overfit. We can also track the loss history in order to visually see the improvement with each iteration.\n", "\n", "Then we can use the trained model to predict whether images from the test dataset are cats or dogs (with a predicted probability for each)."], "metadata": {"_uuid": "34aad348a7a6e849fe6fbca4f8481cc4a1eeb824", "_cell_guid": "b7fbf156-2268-62ce-5fed-52a09af5e6f7"}}, {"cell_type": "code", "execution_count": null, "source": ["from keras.models import Sequential\n", "from keras.layers import Input, Dropout, Flatten, Conv2D, MaxPooling2D, Dense, Activation\n", "from keras.optimizers import RMSprop\n", "from keras.callbacks import ModelCheckpoint, Callback, EarlyStopping\n", "from keras.utils import np_utils\n", "\n", "epochs = 10\n", "batch_size = 16\n", "\n", "## Callback for loss logging per epoch\n", "class LossHistory(Callback):\n", "    def on_train_begin(self, logs={}):\n", "        self.losses = []\n", "        self.val_losses = []\n", "        \n", "    def on_epoch_end(self, batch, logs={}):\n", "        self.losses.append(logs.get('loss'))\n", "        self.val_losses.append(logs.get('val_loss'))\n", "\n", "early_stopping = EarlyStopping(monitor='val_loss', patience=3, verbose=1, mode='auto')        \n", "       \n", "\n", "def run_catdog():\n", "    \n", "    history = LossHistory()\n", "    print(\"running model...\")\n", "    model.fit(train, labels, batch_size=batch_size, epochs=epochs,\n", "              validation_split=0.25, verbose=2, shuffle=True, callbacks=[history, early_stopping])\n", "    \n", "    print(\"making predictions on test set...\")\n", "    predictions = model.predict(test, verbose=0)\n", "    return predictions, history\n", "\n", "predictions, history = run_catdog()\n", "\n", "loss = history.losses\n", "val_loss = history.val_losses\n", "\n", "\n", "plt.xlabel('Epochs')\n", "plt.ylabel('Loss')\n", "plt.title('VGG-16 Loss Trend')\n", "plt.plot(loss, 'blue', label='Training Loss')\n", "plt.plot(val_loss, 'green', label='Validation Loss')\n", "plt.xticks(range(0,epochs)[0::2])\n", "plt.legend()\n", "plt.show()"], "metadata": {"scrolled": false, "_uuid": "1c34f989f9e40e1dcad64ba6fb081c5d64e007ba", "_cell_guid": "ca613169-4b41-e9d5-27c7-6629f8e035c8"}, "outputs": []}, {"cell_type": "markdown", "source": ["## How'd We Do?\n", "\n", "Humans are very good at distinguising cats vs. dogs, but how well does our model perform? Below are some examples from the test data set.\n", "\n", "**to do: create confusion matrix and/or ROC curve**\n"], "metadata": {"_uuid": "8a0d76bbed3c391071f6dbec3a1c8d33d317a959", "_cell_guid": "d9382e7d-31db-0d75-6876-aaccb126e4f6"}}, {"cell_type": "code", "execution_count": null, "source": ["#####predict cat | predict dog\n", "\n", "\n", "for i in range(0,10):\n", "    if predictions[i, 0] >= 0.5: \n", "        print('I am {:.2%} sure this is a Dog'.format(predictions[i][0]))\n", "    else: \n", "        print('I am {:.2%} sure this is a Cat'.format(1-predictions[i][0]))\n", "        \n", "    plt.imshow(test2[i].T)\n", "    plt.show()"], "metadata": {"collapsed": true, "_uuid": "80407ec65acef1f74ff18d121aee58f6052716f0", "_cell_guid": "c665d01b-a5a7-2f7e-e5d4-09e108920e40"}, "outputs": []}], "nbformat_minor": 1}