{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Comparing vision models\n\n![Banner](https://miro.medium.com/max/1920/1*oB3S5yHHhvougJkPXuc8og.gif)\n\nIn this notebook, we are going to compare a few models on the dog vs cats classification. We will *not* be working for some state of the art results or implement some cutting edge technique, but are going to compare a few models. The models will be compared on a set of fixed hyper-parameters which could be tuned and tweak as per the requirement.\n\nWe are going to compare the following models:\n1. [Custom plain convolution model](#Custom-Model)\n2. [VGG-16](#VGG-16)\n3. [VGG-19](#VGG-19)\n3. [ResNet50](#ResNet50)\n4. [ResNet101](#ResNet101)"},{"metadata":{},"cell_type":"markdown","source":"## Unloading the datasets"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"np.random.seed(2020)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"!mkdir train test\n!unzip -q /kaggle/input/dogs-vs-cats-redux-kernels-edition/train.zip -d train\n!unzip -q /kaggle/input/dogs-vs-cats-redux-kernels-edition/test.zip -d test","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"","_uuid":"","trusted":true},"cell_type":"markdown","source":"## Setting up the data\n\nFirst of all, we are going to setup the data paths and files into required variables."},{"metadata":{"trusted":true},"cell_type":"code","source":"import random\nimport json\nimport csv\n\nfrom matplotlib import pyplot as plt\n%matplotlib inline\n\nimport keras\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D\nfrom keras.preprocessing import image\nfrom keras import optimizers","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_DIR = './train/train/'\nTEST_DIR = './test/test'\n\nROWS = 150\nCOLS = 150\nCHANNELS = 3\n\nBATCH_SIZE=64\n\n# HyperParams\nEPOCHS=5\ntrain_steps = len(os.listdir(TRAIN_DIR))/BATCH_SIZE\nvalidation_steps = len(os.listdir(TEST_DIR))/BATCH_SIZE\nlr=1e-4","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"original_train_images = [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR)] # use this for full dataset\ntrain_dogs =   [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR) if 'dog' in i]\ntrain_cats =   [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR) if 'cat' in i]\n\ntest_images =  [TEST_DIR+i for i in os.listdir(TEST_DIR)]\n\n# slice datasets for memory efficiency on Kaggle Kernels, delete if using full dataset\noriginal_train_images = train_dogs[:12000] + train_cats[:12000]\n# test_images =  test_images[:100]\n\n# section = int(len(original_train_images) * 0.8)\ntrain_images = original_train_images[:18000]\nvalidation_images = original_train_images[18000:]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(train_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_arr(arr):\n    plt.figure()\n    plt.imshow(image.array_to_img(arr))\n    plt.show()\n\ndef plot(img):\n    plt.figure()\n    plt.imshow(img)\n    plt.show()\n    \ndef prep_data(images):\n    count = len(images)\n    X = np.ndarray((count, ROWS, COLS, CHANNELS), dtype=np.float32)\n    y = np.zeros((count,), dtype=np.float32)\n    \n    for i, image_file in enumerate(images):\n        img = image.load_img(image_file, target_size=(ROWS, COLS))\n        X[i] = image.img_to_array(img)\n        if 'dog' in image_file:\n            y[i] = 1.\n        if i%1000 == 0: print('Processed {} of {}'.format(i, count))\n    \n    return X, y","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train, y_train = prep_data(train_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X_validation, y_validation = prep_data(validation_images)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_datagen = image.ImageDataGenerator(\n    rescale=1./255,\n    rotation_range=40,\n    width_shift_range=0.2,\n    shear_range=0.2,\n    zoom_range=0.2,\n    horizontal_flip=True)\n\nvalidation_datagen = image.ImageDataGenerator(rescale=1./255)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_generator = train_datagen.flow(\n    X_train,\n    y_train,\n    batch_size=BATCH_SIZE)\n\nvalidation_generator = validation_datagen.flow(\n    X_validation,\n    y_validation,\n    batch_size=BATCH_SIZE)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Custom Model"},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_custom_model():\n    model = Sequential()\n    model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(ROWS, COLS, CHANNELS)))\n    model.add(Conv2D(32, (3, 3), activation='relu'))\n    model.add(MaxPooling2D(pool_size=(2, 2)))\n    model.add(Conv2D(64, (3, 3), activation='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2)))\n    model.add(Conv2D(128, (3, 3), activation='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2)))\n    model.add(Conv2D(128, (3, 3), activation='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2)))\n\n    model.add(Flatten())\n    model.add(Dropout(0.5))\n    model.add(Dense(512, activation='relu'))\n    model.add(Dense(1, activation='sigmoid'))\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = create_custom_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.compile(loss='binary_crossentropy',\n             optimizer=optimizers.Adam(lr=lr),\n             metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    train_generator,\n    steps_per_epoch=train_steps,\n    epochs=EPOCHS,\n    validation_data=validation_generator,\n    validation_steps=validation_steps,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'b', label='Training accuracy')\nplt.plot(epochs, val_acc, 'b', label='Validation accuracy', color='red')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs, loss, 'b', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss', color='red')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## VGG-16"},{"metadata":{},"cell_type":"markdown","source":"VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper “Very Deep Convolutional Networks for Large-Scale Image Recognition”. The model achieves 92.7% top-5 test accuracy in ImageNet, which is a dataset of over 14 million images belonging to 1000 classes. It was one of the famous model submitted to ILSVRC-2014. It makes the improvement over AlexNet by replacing large kernel-sized filters (11 and 5 in the first and second convolutional layer, respectively) with multiple 3×3 kernel-sized filters one after another. \n\n![VGG-16](https://xgkfq28377.i.lithium.com/t5/image/serverpage/image-id/8241i196E2A78143567C5/image-size/medium?v=1.0&px=400)\n\nThe input to cov1 layer is of fixed size 224 x 224 RGB image. The image is passed through a stack of convolutional (conv.) layers, where the filters were used with a very small receptive field: 3×3 (which is the smallest size to capture the notion of left/right, up/down, center). In one of the configurations, it also utilizes 1×1 convolution filters, which can be seen as a linear transformation of the input channels (followed by non-linearity). The convolution stride is fixed to 1 pixel; the spatial padding of conv. layer input is such that the spatial resolution is preserved after convolution, i.e. the padding is 1-pixel for 3×3 conv. layers. Spatial pooling is carried out by five max-pooling layers, which follow some of the conv.  layers (not all the conv. layers are followed by max-pooling). Max-pooling is performed over a 2×2 pixel window, with stride 2.\n\nThree Fully-Connected (FC) layers follow a stack of convolutional layers (which has a different depth in different architectures): the first two have 4096 channels each, the third performs 1000-way ILSVRC classification and thus contains 1000 channels (one for each class). The final layer is the soft-max layer. The configuration of the fully connected layers is the same in all networks.\n\nThe Network architecture is as follows:\n![architecture](https://neurohive.io/wp-content/uploads/2018/11/Capture-564x570.jpg)\n\nHere is a link to the sample VGG network for Keras: \n[Keras code of VGG-16](https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3)"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.Sequential()\nmodel.add(keras.applications.VGG16(include_top=False, pooling='max', weights='imagenet'))\nmodel.add(Dense(1, activation='sigmoid'))\n# ResNet-50 model is already trained, should not be trained\nmodel.layers[0].trainable = True\n\nmodel.compile(loss='binary_crossentropy',\n             optimizer=optimizers.Adam(lr=lr),\n             metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    train_generator,\n    steps_per_epoch=train_steps,\n    epochs=EPOCHS,\n    validation_data=validation_generator,\n    validation_steps=validation_steps,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nloss = history.history['loss']\nval_loss = history.history['val_accuracy']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'b', label='Training accuracy')\nplt.plot(epochs, val_acc, 'b', label='Validation acc', color='red')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs, loss, 'b', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss', color='red')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# VGG-19"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.Sequential()\nmodel.add(keras.applications.VGG19(include_top=False, pooling='max', weights='imagenet'))\nmodel.add(Dense(1, activation='sigmoid'))\n# ResNet-50 model is already trained, should not be trained\nmodel.layers[0].trainable = True\n\nmodel.compile(loss='binary_crossentropy',\n             optimizer=optimizers.Adam(lr=lr),\n             metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    train_generator,\n    steps_per_epoch=train_steps,\n    epochs=EPOCHS,\n    validation_data=validation_generator,\n    validation_steps=validation_steps,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'b', label='Training accuracy')\nplt.plot(epochs, val_acc, 'b', label='Validation accuracy', color='red')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs, loss, 'b', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss', color='red')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The differences between the “VGG-16 Neural Network” and the “VGG-19 Neural Network” are as follows respectively:\n\n1. The “VGG-19 Neural Network” consists of 19 layers of deep neural network whereas the “VGG-16 Neural Network” consists of 16 layers of deep neural network respectively.\n2. The smaller number in terms of deep neural network is used for “ImageNet” and the other bigger number in terms of deep neural network is used for “CIFAR-10” respectively.\n3. The size of the “VGG-16” network in terms of fully connected nodes is 533 MB. and the size of the “VGG-19” network in terms of fully connected nodes is 574 MB. respectively.\n4. The smaller net neural network in terms of “VGG-16” are more desirable like “Squeezenet”, “GoogLeNet” etc. , whereas the more larger net in terms of neural network employs certain deep learning techniques as well as certain image classification problems as well respectively.\n\nSo, this was all about the most basic difference between the “VGG-16” and the “VGG-19” Neural Networks respectively, and what does they actually mean and also what does they stands for and what are their file size and also what does they actually deployed and implemented in terms of various “Neural Networks” respectively.There are many more differences between them, but these were some of the very most basic differences between these two types of the “Neural Networks” respectively."},{"metadata":{},"cell_type":"markdown","source":"## ResNet50"},{"metadata":{},"cell_type":"markdown","source":"> This was one of the bottlenecks of VGG. They couldn’t go as deep as wanted, because they started to lose generalization capability.\n\nTo solve this problem, Resnets were introduced. \nOne of the problems ResNets solve is the famous known vanishing gradient. This is because when the network is too deep, the gradients from where the loss function is calculated easily shrink to zero after several applications of the chain rule. This result on the weights never updating its values and therefore, no learning is being performed.\n\nWith ResNets, **the gradients can flow directly through the skip connections backwards from later layers to initial filters.**\n\n![Resnet](https://miro.medium.com/max/1524/1*6hF97Upuqg_LdsqWY6n_wg.png)\n\nSince ResNets can have variable sizes, depending on how big each of the layers of the model are, and how many layers it has, we will follow the described by the authors in the paper [1] — ResNet 34 — in order to explain the structure after these networks.\n\nIn here we can see that the ResNet (the one on the right) consists on one convolution and pooling step (on orange) followed by 4 layers of similar behavior.\n\nEach of the layers follow the same pattern. They perform 3x3 convolution with a fixed feature map dimension (F) [64, 128, 256, 512] respectively, bypassing the input every 2 convolutions. Furthermore, the width (W) and height (H) dimensions remain constant during the entire layer.\n\nThe dotted line is there, precisely because there has been a change in the dimension of the input volume (of course a reduction because of the convolution). Note that this reduction between layers is achieved by an increase on the stride, from 1 to 2, at the first convolution of each layer; instead of by a pooling operation, which we are used to see as down samplers.\nIn the table, there is a summary of the output size at every layer and the dimension of the convolutional kernels at every point in the structure.\n\n![Resnet Table](https://miro.medium.com/max/936/1*I2557MCaFdNUm4q9TfvOpw.png)\n\nNow in order to calculate the shape of the upcoming block, here is what should be done:\n\n![block](https://miro.medium.com/max/680/1*CJn_fMeW4m2OSt71jzO4WA.png)\n\n![block2](https://miro.medium.com/max/904/1*_kbJ_fvRhVPQ1fRRssEwhA.png)\n\n\n\nTo understand the function of every single block in Residual Network, you should refer to an amazing blog : [Understanding and visualizing ResNets](https://towardsdatascience.com/understanding-and-visualizing-resnets-442284831be8)"},{"metadata":{},"cell_type":"markdown","source":"### Implementing ResNet in Keras"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.Sequential()\nmodel.add(keras.applications.ResNet50(include_top=False, pooling='max', weights='imagenet'))\nmodel.add(Dense(1, activation='sigmoid'))\n# ResNet-50 model is already trained, should not be trained\nmodel.layers[0].trainable = True\n\nmodel.compile(loss='binary_crossentropy',\n             optimizer=optimizers.Adam(lr=lr),\n             metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    train_generator,\n    steps_per_epoch=train_steps,\n    epochs=EPOCHS,\n    validation_data=validation_generator,\n    validation_steps=validation_steps,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'b', label='Training acc')\nplt.plot(epochs, val_acc, 'b', label='Validation acc', color='red')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs, loss, 'b', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss', color='red')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = keras.models.Sequential()\nmodel.add(keras.applications.ResNet101(include_top=False, pooling='max', weights='imagenet'))\nmodel.add(Dense(1, activation='sigmoid'))\n# ResNet-50 model is already trained, should not be trained\nmodel.layers[0].trainable = True\n\nmodel.compile(loss='binary_crossentropy',\n             optimizer=optimizers.Adam(lr=lr),\n             metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit_generator(\n    train_generator,\n    steps_per_epoch=train_steps,\n    epochs=EPOCHS,\n    validation_data=validation_generator,\n    validation_steps=validation_steps,\n    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nepochs = range(1, len(acc) + 1)\nplt.plot(epochs, acc, 'b', label='Training accuracy')\nplt.plot(epochs, val_acc, 'b', label='Validation accuracy', color='red')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs, loss, 'b', label='Training loss')\nplt.plot(epochs, val_loss, 'b', label='Validation loss', color='red')\nplt.title('Training and validation loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"!rm -rf train\n!rm -rf test","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}