{"cells":[{"metadata":{"_uuid":"a7e1407d19fd227295c6fd016b9e77db1247651c"},"cell_type":"markdown","source":"#Dogs vs. Cats Updated"},{"metadata":{"_uuid":"1c1703ec5f83e3d8b5e2a9b6ec584c7b3d2d6f20"},"cell_type":"markdown","source":"This Kernel will expand on my previous Kernel that used a Convolutional Neural Network (CNN) to predict pictures of Dogs and Cats. Most of the code that reads in and preprocesses the data was discussed in my previous kernel (https://www.kaggle.com/kasmithh/dogs-vs-cats) and is based on code from the following tutorial (https://www.youtube.com/watch?v=gT4F3HGYXf4&t=554s). After reading in and preprocessing the data, this Kernel will address the overfitting proplem that we had in the previous Kernel by using dropout layers, more convolutional layers and implementing a batch size during the training process. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, MaxPool2D, Conv2D, Flatten\nimport cv2\nfrom random import shuffle\nimport matplotlib.pyplot as plt\n\nimport os\n\nimg_path = '../input/train'\nIMG_SIZE = 100","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"def label_img(img):\n    word_label = img.split('.')[0]\n    if word_label == 'cat':\n        return [1,0]\n    if word_label == 'dog':\n        return [0,1]\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6a0a79020dde93ce6eb927189b86ee59065d2db3"},"cell_type":"code","source":"def create_train_data():\n    training_data = []\n    for img in os.listdir(img_path):\n        label = label_img(img)\n        path = os.path.join(img_path,img)\n        img = cv2.resize(cv2.imread(path, cv2.IMREAD_GRAYSCALE), (IMG_SIZE, IMG_SIZE))\n        training_data.append([np.array(img), np.array(label)])\n        shuffle(training_data)\n    return training_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"23674f4877f4ffa3da6bb42e3eeccdf4005a08ad"},"cell_type":"code","source":"train = create_train_data()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f087762cdacac7090abfa33e9761b71791b74961"},"cell_type":"code","source":"X = np.array([i[0] for i in train])\nX = X.reshape(-1,100,100,1)\nX = X/255\nY = np.array([i[1] for i in train])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"77b92fd09ced0a7c90c6c22e3bd3fbcb9f8c3f59"},"cell_type":"markdown","source":"##Addressing Overfitting"},{"metadata":{"_uuid":"b035bbdc58c09b5f44e6ff7bfded9a42c7be2726"},"cell_type":"markdown","source":"In order to address overfitting and improve the model from the previous Kernel we will use a series of techniques that include adding more convolutional layers, dropout layers, and specifying a batch size during the training process. Dropout layers set a specified rate of inputs to 0 during the training process or in other words nodes of the next work are \"dropped out\". This allows the model to generalize easier instead of just learning the training data. More on dropout can be found here: (https://keras.io/layers/core/#dropout). Batch size determines how many samples from the training set will be used to train the model at a time. So in our case with a batch size of 32, 32 samples from the training set will be used to train the model at a time. (https://stats.stackexchange.com/questions/153531/what-is-batch-size-in-neural-network)"},{"metadata":{"_uuid":"32619a64a5ffa16ef6dbbb4464b344ebf3efdd7b"},"cell_type":"markdown","source":"The code for creating the CNN should look very similar to the previous model, but this time the model includes more convolutional layers and a couple of dropout layers. Each dropout layer is set to 30% meaning that 30% of the nodes will be set to 0."},{"metadata":{"trusted":true,"_uuid":"9d61d8d69eb18bcc94956660aa46823c5bac3da9"},"cell_type":"code","source":"model = Sequential()\nmodel.add(Conv2D(64, kernel_size = (3,3), input_shape=(100,100,1), activation = 'relu'))\nmodel.add(Conv2D(64, kernel_size = (3,3), activation = 'relu'))\nmodel.add(Dropout(0.30))\nmodel.add(MaxPool2D(2,2))\nmodel.add(Conv2D(64, kernel_size = (3,3), activation = 'relu'))\nmodel.add(Conv2D(64, kernel_size = (3,3), activation = 'relu'))\nmodel.add(Dropout(0.30))\nmodel.add(MaxPool2D(2,2))\nmodel.add(Flatten())\nmodel.add(Dense(64, activation = 'relu'))\nmodel.add(Dropout(0.30))\nmodel.add(Dense(2, activation = 'softmax'))\nmodel.compile(optimizer='adam',\n              loss= 'binary_crossentropy',\n              metrics=['accuracy', 'mae'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"31daa0958df3942423d3cd8dbfe56a235d5e2d8f"},"cell_type":"markdown","source":"During the fitting process is where we establish our batch size of 32."},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"2949ce07bc0d6e9826d1adc89990e4e615a3f151"},"cell_type":"code","source":"Fit = model.fit(X, Y, epochs = 5, batch_size = 32, validation_split = 0.30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2459a03c214c98a0a934de91036bd14f377b3027"},"cell_type":"markdown","source":"As we can see in the graphs below by implementing dropout layers, a batch size during training, and more convolutional layers, the model is preforming about as well as the previous model in terms of validation accuracy and loss, but is not overfitting to the training data. Validation accuracy and loss are a lot closer to that of the training data this time unlike in the first model where the training accuracy and loss were very different than that of the validation accuracy and loss."},{"metadata":{"trusted":true,"_uuid":"b5abe985e77bbff532604110ec9d529a7612dba0"},"cell_type":"code","source":"plt.plot(Fit.history['loss'])\nplt.plot(Fit.history['val_loss'])\nplt.ylabel('loss')\nplt.xlabel('epochs')\nplt.legend(['train', 'validation'], loc = 'upper left')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"517a90c75f8d91971b64b1fe8d709ea2105419a3"},"cell_type":"code","source":"plt.plot(Fit.history['acc'])\nplt.plot(Fit.history['val_acc'])\nplt.ylabel('Accuracy')\nplt.xlabel('epochs')\nplt.legend(['train', 'validation'], loc = 'upper left')\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}