{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Dogs vs Cats classification using transfer learning and CNN"},{"metadata":{},"cell_type":"markdown","source":"This kernel shows how to use transfer learning to classify dogs and cats images. If you find this kernel useful please upvote it!\n\nTransfer learning means that instead of your model learning everything from scratch, it uses another model that was trained on a similar problem, so that you can \"transfer\" the learned \"knowledge\" of the pretrained model to your model, and then learn some new features.\n\nThe ImageNet Data set is huge data set consisting of more that 14 million images from more than 22,000 different categories, here we are using a smaller version of it which has 1000 different categories.\n\nIn this kernel we use an xception model which is pretrained on the ImageNet dataset and then build some layers on top of it to be able to classify dogs and cats images.\n\nTransfer learning makes sense here because the ImageNet data set has a much larger number of images (14 million) than the dog-cat data set (25,000). This increases the speed of training of our model and the accuracy of our predictions."},{"metadata":{},"cell_type":"markdown","source":"# Import Required Libraries\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nimport random\nimport os\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport keras.applications.xception as xception\nimport zipfile\nimport sys\nimport time\nimport tensorflow.keras as keras\n\nfrom PIL import Image\nfrom keras.layers import Input, Conv2D, Dense, Flatten, MaxPooling2D, Input\nfrom keras.models import Model, Sequential\nfrom keras.preprocessing import image\nfrom keras.utils import to_categorical\n\nprint('setup successful!')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define Constants"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"IMAGE_WIDTH = 224    \nIMAGE_HEIGHT = 224\nIMAGE_CHANNELS = 3\n\n# The image net have 1000 categories, and so 1000 outputs, so we get 1000 different features\nIMAGE_FEATURES_SIZE = 1000\n\n# For the first run PERFORM_UNZIPPING must be True, but after the first run, \n#it can be set to False skip the unzipping step to save time\nPERFORM_UNZIPPING = True \n\nprint('defining constants successful!')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Unzip the Train and Test folders\n\nThe train and test folders are zipped, the code below unzips both folders and the unzipped folders are saved in the output folder. For the First run PERFORM_UNZIPPING should be true to perform the unzipping, later if the folders are already unzipped you can change it to False"},{"metadata":{"trusted":true},"cell_type":"code","source":"if PERFORM_UNZIPPING == True:\n    #Unzip the input folders\n    with zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/train.zip\",\"r\") as z:\n        z.extractall(\".\")        \n    \n    with zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/test.zip\",\"r\") as z:\n        z.extractall(\".\")   \n    \n    print('Unzipping done!')\n    \n    \nelse:\n    print('Unzipping not needed')\n    \n    \nimgs_path = \"/kaggle/working/train/\"\nrealtestimgs_path = \"/kaggle/working/test/\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prepare load_image function and Data Frame\nThe pretrained model that we are using here is the xception model. The advantage of the xception model is that it has a small size, relatively small number of parameters and high accuracy. You can have a look with on a comparison of different pretrained models available in Keras in https://keras.io/api/applications/\n\n\nThe function 'load_image' takes the path of an image as an input and return the preprocessed image. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"def _load_image(img_path):\n    img = image.load_img(img_path, target_size = (IMAGE_WIDTH, IMAGE_HEIGHT)) # load the image from the directory\n    img = image.img_to_array(img) \n    # add an additional dimension, (i.e. change the shape of each image from (224, 224, 3) to (1, 224, 224, 3)\n    # This shape is suitable for training\n    img = np.expand_dims(img, axis = 0) \n    # Apply preprocessing for the image, so that the training is faster\n    img = xception.preprocess_input(img)\n        \n    return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Create a  dataframe that contains a list of the file names and the corresponding category. Category 1 means dog, 0 means cat"},{"metadata":{"trusted":true},"cell_type":"code","source":"filenames = os.listdir(imgs_path)\n\ncategories = []\n\nfor filename in filenames:\n    category = filename.split('.')[0]\n    if category == 'dog':\n        categories.append(1)\n    else:\n        categories.append(0)\n\ndf = pd.DataFrame({\n    'filename': filenames,\n    'category': categories\n})\n\nprint(df.head())\n\nprint('number of elements = ' , len(df))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# see sample image, you can run the same cell again to get a different image\nsample = random.choice(filenames)\nrandomimage = image.load_img(imgs_path + sample)\nplt.imshow(randomimage)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Create the Pretrained Model\nThere are two approaches for transfer learning:\n1. One model where the first part of it is the pretrained model and the second part is your new model. \n1. Pass all the images through the pretrained model, to get the extracted features and then use those extrated features to    train your new model. \n\nHere we will use the second approach (the feature extraction approach), because a matrix which contains all our images with the shape (224, 224, 3) will not fit in our RAM. So we first pass all the images through the xception model and get the extracted features, as mentioned above the model has 1000 categories, so the output will have a size of 1000 which is much smaller than (224 X 224 X 3 = 150,528) . So if we have 100 images the tarining matrix will have a shape of (100, 1000)"},{"metadata":{"trusted":true},"cell_type":"code","source":"# create the xception model used for the feature extraction\nmodel_xception = xception.Xception(include_top = True, input_shape = (IMAGE_WIDTH, IMAGE_HEIGHT,IMAGE_CHANNELS),\n                       weights = '../input/xception/xception_weights_tf_dim_ordering_tf_kernels.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will use the EarlyStopping call back to stop our training  if the accuracy is not improving for a certain number of epochs. \nIf the accuracy is not improving for a certain number of epochs then it makes sense to reduce the learning rate, we implement that using the ReduceLROnPlateau call back."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define call backs\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau\n\nearly_stop = EarlyStopping(patience = 7, verbose = 1)\n\nlearning_rate_reduction = ReduceLROnPlateau(patience = 3, verbose = 1, factor=0.5)\n\nprint('call backs defined!')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Feature Extraction\nThe extract_features function will take the pretrained model used for feature extraction and a batch of images, then it will pass those images through the model to obtain the extracted features. It is better to pass a maximum of a round 1000 images at a time in order not to fill our RAM. The data_set parameter is dataframe containing filenames and the corresponding categories for each file."},{"metadata":{"trusted":true},"cell_type":"code","source":"def extract_features(model, data_set):\n    \n    loaded_images = np.zeros((len(data_set), IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS))\n        \n    for i, data_set_entry in enumerate (data_set):\n        file_name = data_set_entry[0]\n        category  = data_set_entry[1]\n        path = imgs_path + file_name        \n        loaded_img = _load_image(path)\n        loaded_images[i, :, :, :] = loaded_img\n\n    preds = model.predict(loaded_images)   \n    \n    return preds","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We split the training set into three separate sets:\n1. **The training set**: used to train our model.\n2. **The validation set**: used to double check that our model is not overfitting the training set, i.e. it can also generalise    to other data other than the train data\n3. **The Test set**: Used to estimate the accuracy of the model on new data other than the ones the model used for training\n\nFor a competetion or for some other cases, you can split the data only to training and validation sets inorder to acheive te highest possible accuracy, without the need to properly estimate how accurate the model really is.\n\nWe split the data set as follows: 2% test set, 82% train set, 16% cross_validation set\n \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_set, validate_set, test_set = np.split(df.sample(frac=1, random_state =43), [int(.82*len(df)), int(.96*len(df))])\n\n# Visualize the data distribution\nfig, axis = plt.subplots(1,3)\nfig.tight_layout()\nfig.set_size_inches(80,20)\n\nax0 = sns.countplot(data=train_set   , x = 'category', ax = axis[0])\nax1 = sns.countplot(data=validate_set, x = 'category', ax = axis[1])\nax2 = sns.countplot(data=test_set    , x = 'category', ax = axis[2])\n\nax0.set_title('train data', fontsize = 60)\nax1.set_title('cross validation data', fontsize = 60)\nax2.set_title('test data', fontsize = 60)\n\nax0.tick_params(labelsize=55)\nax1.tick_params(labelsize=55)\nax2.tick_params(labelsize=55)\n\nprint('shape of train_set:   ', np.shape(train_set))\nprint('shape of validate_set:', np.shape(validate_set))\nprint('shape of test_set:    ', np.shape(test_set))\n\n\n# convert the dataframes to numpy matrices\ntrain_set    = train_set.to_numpy()\nvalidate_set = validate_set.to_numpy()\ntest_set     = test_set.to_numpy()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we perform the feature extraction. We take chunks of the train_set (1000 at a time) and pass them through the pretrained model, and save the extracted features in train_x. We then do the same with the validation_set."},{"metadata":{"trusted":true},"cell_type":"code","source":"def chunks(lst, n):\n    \"\"\"Yield successive n-sized chunks from lst.\"\"\"\n    for i in range(0, len(lst), n):\n        yield lst[i:i + n]\n        \n# prepare parameters for model.fit\ntrain_x = np.zeros((len(train_set), 1000)) \ntrain_y = np.zeros((len(train_set)))\n\nvalidate_x = np.zeros((len(validate_set), 1000)) \nvalidate_y = np.zeros((len(validate_set) ))\n\nchunk_size = 1000\n\n# extract features from train_set nd save it into train_x\nfor i, train_set_chunk in enumerate (chunks(train_set, chunk_size)):\n    train_x[ (i*chunk_size) : (i*chunk_size + chunk_size)] = extract_features(model_xception, train_set_chunk)    \n\nprint('shape of train_x: ',    np.shape(train_x))\n\n\n# extract features from validate_set nd save it into validate_x\nfor i, validate_set_chunk in enumerate (chunks(validate_set, chunk_size)):\n    validate_x[ (i*chunk_size) : (i*chunk_size + chunk_size)] = extract_features(model_xception, validate_set_chunk)    \n\nprint('shape of validate_x: ', np.shape(validate_x))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We take the second column from the train_set (the category column) and save it in train_y. We then convert it to a one-hot vector using the \"to_categorically\" method, we need the one-hot format for the model fitting later on."},{"metadata":{"trusted":true},"cell_type":"code","source":"# prepare train_y\ntrain_y = train_set[:,1]\ntrain_y = to_categorical(train_y)        \n\n# prepare validate_y\nvalidate_y = validate_set[:,1]\nvalidate_y = to_categorical(validate_y)   ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the new model and train it\nNote that it is a small model, that is because it doesn't need to learn everything from scratch because we are using the extracted features. Also learning will take just around 30 seconds using the GPU."},{"metadata":{"trusted":true},"cell_type":"code","source":"transfer_model = Sequential()\n\ntransfer_model.add(keras.Input(shape = (IMAGE_FEATURES_SIZE)))\ntransfer_model.add(Flatten())\ntransfer_model.add(Dense(100, activation = 'relu'))\ntransfer_model.add(Dense(16,  activation = 'relu'))\ntransfer_model.add(Dense(2,   activation = 'softmax')) # The last layer has 2 units beacuse we have 2 classes (Dog and cat)\n\ntransfer_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# compile and fit the model\ntransfer_model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])\n\nEPOCHS = 20\n\nhistory = transfer_model.fit(x = train_x , y= train_y, batch_size = 16, epochs = EPOCHS, \n                             callbacks = [early_stop, learning_rate_reduction],\n                             validation_data = (validate_x, validate_y))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"transfer_model.save_weights(\"model.h5\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualize the training process"},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 10))\nax1.plot(history.history['loss'], color='b', label=\"Training loss\")\nax1.plot(history.history['val_loss'], color='r', label=\"validation loss\")\nax1.set_xticks(np.arange(1, EPOCHS, 1))\nax1.set_yticks(np.arange(0, 0.14, 0.02))\nax1.legend()\n\nax2.plot(history.history['accuracy'], color='b', label=\"Training accuracy\")\nax2.plot(history.history['val_accuracy'], color='r',label=\"Validation accuracy\")\nax2.set_xticks(np.arange(1, EPOCHS, 1))\nax2.legend()\n\nlegend = plt.legend(loc='best')\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Predict\nPredict the class for the test set and evaluate the accuracy on the test set."},{"metadata":{"trusted":true},"cell_type":"code","source":"test_x = np.zeros((len(test_set), 1000)) \ntest_y = np.zeros((len(test_set) ))\n\n# extract features from test_set and save it into test_x\nfor i, test_set_chunk in enumerate (chunks(test_set, chunk_size)):\n    test_x[ (i*chunk_size) : (i*chunk_size + chunk_size)] = extract_features(model_xception, test_set_chunk)    \n\n# prepare test_y\ntest_y = test_set[:,1]\ntest_y = to_categorical(test_y) \n\nassert(np.shape(test_y)  == (np.shape(test_set)[0], 2))\n\n# predict and estimate the accuracy\n_, accuracy = transfer_model.evaluate(test_x, test_y)   \nprint('accuracy on test set = ',  round((accuracy * 100),2 ), '% ') ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Predict Class of Random Image\nThe cell below will randomly select an image from the test set, predict if it is a dog or cat. You can run the cell several times to get different images."},{"metadata":{"trusted":true},"cell_type":"code","source":"sample = random.choice(test_set)[0]\nrandomimage = image.load_img(imgs_path + sample)\nplt.imshow(randomimage)\n\nloaded_image = _load_image(imgs_path + sample)\nextracted_feat = model_xception.predict(loaded_image) \npred = transfer_model.predict(extracted_feat)\n\npred = pred[0]  # convert to array\nif pred[0] >= pred[1]:\n    prediction_class = 'cat'\n    prediction_percentage = pred[0]\nelse:\n    prediction_class = 'dog'\n    prediction_percentage = pred[1]\n\nprint('I am ',int(prediction_percentage*100), '% sure that I am a ', prediction_class, '!', sep = '')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's it, if you find this kernel useful please upvote it :)\n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}