{"cells":[{"metadata":{"_uuid":"3d57ebbee25c593221d8c8f3c0bd4bb3c0c7dbd8"},"cell_type":"markdown","source":"# Dogs vs. Cats Classification\n\n## Data Analysis:\nThe data provided contains two folders: \n1. train.zip - Contains 25,000 images of cats and dogs and should be used for training the model we build.\n2. test1.zip - Contains 12,500 images which must be classified into dogs or cats based on the information we collect and the model we build. \n\n### Train Data\nIn the training data, we observe that the name of each image contains information regarding which class that image belongs to. For example, the file \"cat.001.jpg\", belongs to the class \"cat\" and the file \"dog.001.jpg\" belongs to the class \"dog\".\n\n### Test Data\nIn the test data, the name of the files do not contain any information regarding the content of that file and it is our job to predict which class(cat/dog) the image belongs to.\n\n# So, why Convolutional Neural Networks?\n\nWhy CNNs and why not a vanilla neural network? \n\nThe general applicability of neural networks is one of their advantages, but this advantage turns into a liability when dealing with images. The convolutional neural networks make a conscious tradeoff: if a network is designed for specifically handling the images, some generalizability has to be sacrificed for a much more feasible solution.\n\nIf you consider any image, proximity has a strong relation with similarity in it and convolutional neural networks specifically take advantage of this fact. This implies, in a given image, two pixels that are nearer to each other are more likely to be related than the two pixels that are apart from each other. Nevertheless, in a usual neural network, every pixel is linked to every single neuron. Regular Neural Nets don’t scale well to full images. In CIFAR-10, images are only of size 32x32x3 (32 wide, 32 high, 3 color channels), so a single fully-connected neuron in a first hidden layer of a regular Neural Network would have 32*32*3 = 3072 weights. This amount still seems manageable, but clearly this fully-connected structure does not scale to larger images. For example, an image of more respectable size, e.g. 200x200x3, would lead to neurons that have 200*200*3 = 120,000 weights. Moreover, we would almost certainly want to have several such neurons, so the parameters would add up quickly! Clearly, this full connectivity is wasteful and the huge number of parameters would quickly lead to overfitting.\n\nBy killing a lot of these less significant connections, convolution solves this problem. In technical terms, convolutional neural networks make the image processing computationally manageable through filtering the connections by proximity. In a given layer, rather than linking every input to every neuron, convolutional neural networks restrict the connections intentionally so that any one neuron accepts the inputs only from a small subsection of the layer before it(say like 5*5 or 3*3 pixels). Hence, each neuron is responsible for processing only a certain portion of an image.(Incidentally, this is almost how the individual cortical neurons function in your brain. Each neuron responds to only a small portion of your complete visual field).\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\n#We see the contents of the root directory\nprint(\"Root directory contains: \")\nprint(os.listdir(\"../input\"))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"901dcac658dfe01983286a75ded1d9aaf9e02254"},"cell_type":"markdown","source":"# Importing dependencies\nWe use the following packages:\n1. Matplotlib - to plot the training and validation accuracy/loss graphs. Matplotlib is extremely effective for visualize any aspect of the data.\n2. tqdm - used for showing the progress in your loops\n3. TensorFlow - a gradient-based deep learning library that works with Python and has the capability of using GPUs for computation\n4. Keras - a deep learning library capable of running on top of TensorFlow."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom tqdm import tqdm\nimport cv2\nimport tensorflow as tf\nfrom keras.datasets import mnist\nfrom keras import models\nfrom keras import layers\nfrom keras import optimizers\nfrom keras import Sequential\nfrom keras.layers import Dense,MaxPooling2D,Conv2D,Flatten,Dropout, Activation, BatchNormalization\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6a4e2f75a4434faecc5f736a6ad694346b7172d0"},"cell_type":"code","source":"#Declaring the path to the train and test dat\ntrain_path = '../input/train/train'\ntest_path = '../input/test1/test1'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7a2896ce84c87b0f077998c8f0d3f6dbda9d40a0"},"cell_type":"markdown","source":"We create explicit labels for the training data by obtaining the substring from the file name.\nIf a file name starts with \"cat\", we assign it to class \"0\"\nIf a file name starts with \"dog\", we assign it to class \"1\""},{"metadata":{"trusted":true,"_uuid":"66fbef36e2b1e8500fdc9549bec01996aad3fa64"},"cell_type":"code","source":"#Initialize two lists for the data and labels respectively\nlabel=[]\ndata=[]\n\n#Loop iterating over each file in the training folder\nfor file in tqdm(os.listdir(train_path)):\n    #Reading every image and converting it to grayscale\n    image=cv2.imread(os.path.join(train_path,file), cv2.IMREAD_GRAYSCALE)\n    #Resizing the image into a manageable size\n    image=cv2.resize(image,(96,96))\n    #If a file name starts with \"cat\"\n    if file.startswith(\"cat\"):\n        label.append(0)\n    elif file.startswith(\"dog\"):\n        label.append(1)\n    try:\n        data.append(image/255) \n    except:\n        label=label[:len(label)-1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3f73afd09dbc94327acb124e5ab44958f088eab1"},"cell_type":"code","source":"#Converting our data and labels into numpy arrays\ntrain_data=np.array(data)\ntrain_labels=np.array(label)\n\nprint (train_data.shape)\nprint (train_labels.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ce2c15744346e2b0295a884b583415aef553c401"},"cell_type":"code","source":"#Displaying the first image along with the class it belongs to\n\nplt.imshow(train_data[0], cmap='gray')\nplt.title('Class '+ str(train_labels[0]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"093a07ff27e4f5d6d72c7fcdb91e994cce5d0ac5"},"cell_type":"code","source":"#Reshaping our data from a 96x96 array into a 96,96,1 array\ntrain_data = train_data.reshape((train_data.shape)[0],(train_data.shape)[1],(train_data.shape)[2],1)\nprint(train_data.shape)\nprint(train_labels.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b6823b11f27225ee82d79db504bd43b25be58950"},"cell_type":"markdown","source":"## General Architecture of a CNN\nINPUT -  [96x96x1] will hold the raw pixel values of the image, in this case an image of width 96, height 96.\n\nCONV - layer will compute the output of neurons that are connected to local regions in the input, each computing a dot product between their weights and a small region they are connected to in the input volume. This may result in volume such as [96x96x12] if we decided to use 12 filters.\n\nRELU - This layer will apply an elementwise activation function, such as the max(0,x) thresholding at zero. This leaves the size of the volume unchanged ([96x96x12]).\n\nPOOL - This layer will perform a downsampling operation along the spatial dimensions (width, height), resulting in volume such as [16x16x12].\n\nFC - (i.e. fully-connected) layer will compute the class scores, resulting in volume of size [1x1x10], where each of the 10 numbers correspond to a class score, such as among the 10 categories of CIFAR-10. As with ordinary Neural Networks and as the name implies, each neuron in this layer will be connected to all the numbers in the previous volume.\n\n## My Model\nTotal number of layers = 11\n\nInput - [96x96x1]\n\nLayer 1 - Convolution layer with 32 filters , kernel size (3,3) and an activation function as RELU.\n\nLayer 2 - Convolution layer with 64 filters, kernel size (3,3), padding = same and activation function as RELU.\n\nLayer 3 - Max Pooling layer with pool size = (5,5) and strides = (2,2)\n\nLayer 4 - Convolution layer with 10 filters and, kernel size (3,3) and an activation of RELU.\n\nLayer 5 - Convolution layer with 5 filters, kernel size (3,3) and an activation function of RELU.\n\nLayer 6 - Max Pooling layer with pool size = (3,3) and strides = (2,2)\n\nLayer 7 - Convolution layer with 10 filters, kernel size = (2,2) and strides  = 2\n\nLayer 8 - Flatten layer to flatten the vector\n\nLayer 9 - Dropout layer with a dropout rate of 30%. A random 30% of the pixels are initialized to zero.\n\nLayer 10 - Fully connected layer with 100 nodes and activation as Sigmoid\n\nLayer 11 - Output layer with Sigmoid Activation"},{"metadata":{"trusted":true,"_uuid":"a486f7c8e63705823d01c5a3a8b678f7d62aa1eb"},"cell_type":"code","source":"#Creating the model\nmodel = Sequential()\ninput_shape = (96,96,1)\nmodel.add(Conv2D(kernel_size=(3,3),filters=32,input_shape=input_shape,activation=\"relu\"))\nmodel.add(Conv2D(kernel_size=(3,3),filters=64,activation=\"relu\",padding=\"same\"))\nmodel.add(MaxPooling2D(pool_size=(5,5),strides=(2,2)))\n\nmodel.add(Conv2D(kernel_size=(3,3),filters=10,activation=\"relu\"))\nmodel.add(Conv2D(kernel_size=(3,3),filters=5,activation=\"relu\"))\n\nmodel.add(MaxPooling2D(pool_size=(3,3),strides=(2,2)))\n\nmodel.add(Conv2D(kernel_size=(2,2),strides=(2,2),filters=10))\n\nmodel.add(Flatten())\n\nmodel.add(Dropout(0.3))\n\nmodel.add(Dense(100,activation=\"sigmoid\"))\nmodel.add(Dense(1,activation=\"sigmoid\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f1fd83cabba6b843195f350a9e0b598b722624c0"},"cell_type":"code","source":"\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"70d7638f50e9162cfd5f2a0818b7965dada7308c"},"cell_type":"code","source":"#We use the ADADELTA optimization on the binary crossentropy loss function\nmodel.compile(optimizer=\"adadelta\",loss=\"binary_crossentropy\",metrics=[\"accuracy\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3110215e5de9669b62f6a535b52af28c6e0f2bff"},"cell_type":"code","source":"# We try out the model on the training data.\n# Train data has been split. 25% of the training data has been kept aside for Validation. \n# We run the fit function for 20 epochs\n\nmodel_history = model.fit(train_data,train_labels,validation_split=0.25,epochs=20,batch_size=10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"40cc2cc56e00bf54ad115268a61d336fa666e7fd"},"cell_type":"code","source":"#Visualizing accuracy and loss of training the model\nhistory_dict=model_history.history\n\n#Test Accuracy\ntrain_acc = history_dict['acc']\n#Validation Accuracy\nval_acc = history_dict['val_acc']\n\nepochs =range(1,len(train_acc)+1)\n#Plottig the training and validation loss\nplt.plot(epochs, val_acc, 'bo', label='Validation Accuracy')\nplt.plot(epochs, train_acc, 'b', label='Train Accuracy')\nplt.title('Train and Validation Accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"42f048327ae4571d7390c1f0f4ee9e7f80307bf5"},"cell_type":"code","source":"#Training loss\ntrain_loss = history_dict['loss']\n#Validation Loss\nval_loss = history_dict['val_loss']\n\nepochs =range(1,len(train_loss)+1)\n#Plottig the training and validation loss\nplt.plot(epochs, val_loss, 'bo', label='Validation Loss')\nplt.plot(epochs, train_loss, 'b', label='Training Loss')\nplt.title('Train and Validation Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8dd2e0241d708cdaf7edd2425ed976d7a08336b5"},"cell_type":"code","source":"test_data=[]\nid=[]\nfor file in tqdm(os.listdir(test_path)):\n    image_data=cv2.imread(os.path.join(test_path,file), cv2.IMREAD_GRAYSCALE)\n    try:\n        image_data=cv2.resize(image_data,(96,96))\n        test_data.append(image_data/255)\n        id.append((file.split(\".\"))[0])\n    except:\n        print(\"\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5664482e020ff38eb8ea00b5cc8065bf2f1595bc"},"cell_type":"code","source":"test_data1=np.array(test_data)\nprint (test_data1.shape)\ntest_data1=test_data1.reshape((test_data1.shape)[0],(test_data1.shape)[1],(test_data1.shape)[2],1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7d64bc8d0b70d9c71a56350518ef6e85c90ca2cd"},"cell_type":"code","source":"dataframe_output=pd.DataFrame({\"id\":id})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3fe87f667df1ce898a7f4eb5713af034c68e0e31"},"cell_type":"code","source":"predicted_labels=model.predict(test_data1)\npredicted_labels=np.round(predicted_labels,decimals=2)\nprint(predicted_labels)\nlabels=[1 if value>0.5 else 0 for value in predicted_labels]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a37f725166997ec7328635133d5a2ce0247737bb"},"cell_type":"code","source":"dataframe_output[\"label\"]=labels\ndataframe_output.to_csv(\"submission.csv\",index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}