{"cells":[{"metadata":{"id":"b7OkxS36u3_C"},"cell_type":"markdown","source":"## References\n\n- The competition is https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition (it ended long time ago)\n- This implementation is based on https://www.tensorflow.org/tutorials/images/classification","execution_count":null},{"metadata":{"id":"upt6zdESis-1"},"cell_type":"markdown","source":"## Rearrange file structure\n\nTo make it fit to Keras [flow_from_directory](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator#flow_from_directory). See [this article](https://medium.com/@vijayabhaskar96/tutorial-image-classification-with-keras-flow-from-directory-and-generators-95f75ebe5720) for more details. It is assumed that this script runs on Google Colab which mounts Google Drive and have `/Data Set/<data>.zip` below it. You can find the zip file [here](https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition/data).","execution_count":null},{"metadata":{"id":"r1Gm9UbE2HVS","trusted":false},"cell_type":"code","source":"!unzip '/content/drive/My Drive/Data Set/dogs-vs-cats-redux-kernels-edition.zip'\n!unzip 'test.zip'\n!unzip 'train.zip'\n\n!mkdir train/{dog,cat}\n!find train -maxdepth 1 -type f | grep 'dog' | xargs -I{} mv {} train/dog\n!find train -maxdepth 1 -type f | grep 'cat' | xargs -I{} mv {} train/cat\n\n!mkdir test/images\n!find test -maxdepth 1 -type f | xargs -I{} mv {} test/images","execution_count":null,"outputs":[]},{"metadata":{"id":"BtnC7L1DjJJe"},"cell_type":"markdown","source":"## Import libraries","execution_count":null},{"metadata":{"id":"EIYPRiPDTI71","trusted":false},"cell_type":"code","source":"import tensorflow as tf\n\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\n\nimport os\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image\n\nimport pathlib","execution_count":null,"outputs":[]},{"metadata":{"id":"SQkSsiGQjRUR"},"cell_type":"markdown","source":"## Define helper functions and global variables","execution_count":null},{"metadata":{"id":"tmtGxcf9siJw","trusted":false},"cell_type":"code","source":"# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.\ndef plotImages(images_arr):\n    fig, axes = plt.subplots(1, 5, figsize=(20,20))\n    axes = axes.flatten()\n    for img, ax in zip( images_arr, axes):\n        ax.imshow(img)\n        ax.axis('off')\n    plt.tight_layout()\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"dSzzuHElr78Z","trusted":false},"cell_type":"code","source":"train_dir = './train'\ntotal_train = len(os.listdir('./train/dog')) + len(os.listdir('./train/cat'))\ntotal_test = len(os.listdir('./test/images/'))\nvalidation_split = 0.2\n\nbatch_size = 128\nepochs = 10\nIMG_HEIGHT = 150\nIMG_WIDTH = 150","execution_count":null,"outputs":[]},{"metadata":{"id":"wtcYHgZYjbIQ"},"cell_type":"markdown","source":"## Build model and train","execution_count":null},{"metadata":{"id":"sfLTdYccJZjx","outputId":"ab837ee9-3cba-488a-e6af-cd1514c654c8","trusted":false},"cell_type":"code","source":"train_image_generator = ImageDataGenerator(rescale=1./255,\n                                           validation_split=validation_split)\ntrain_data_gen = train_image_generator.flow_from_directory(directory=train_dir,\n                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),\n                                                           class_mode='binary',\n                                                           batch_size=batch_size,\n                                                           shuffle=True,\n                                                           subset='training')\nval_data_gen = train_image_generator.flow_from_directory(directory=train_dir,\n                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),\n                                                           class_mode='binary',\n                                                           batch_size=batch_size,\n                                                           shuffle=True,\n                                                           subset='validation')","execution_count":null,"outputs":[]},{"metadata":{"id":"Lrm9Spxcqkut","trusted":false},"cell_type":"code","source":"model = Sequential([\n    Conv2D(32, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)),\n    Conv2D(32, 3, padding='same', activation='relu'),\n    MaxPooling2D(),\n\n    Conv2D(64, 3, padding='same', activation='relu'),\n    Conv2D(64, 3, padding='same', activation='relu'),\n    MaxPooling2D(),\n\n    Conv2D(128, 3, padding='same', activation='relu'),\n    Conv2D(128, 3, padding='same', activation='relu'),\n    MaxPooling2D(),\n\n    Flatten(),\n    Dense(256, activation='relu'),\n    Dropout(0.5),\n    Dense(256, activation='relu'),\n    Dropout(0.5),\n    Dense(1, activation='sigmoid')\n])","execution_count":null,"outputs":[]},{"metadata":{"id":"X8p2Oh8Lqn_F","trusted":false},"cell_type":"code","source":"model.compile(optimizer='adam',\n              loss='binary_crossentropy',\n              metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"id":"2mvjN6z2qmbt","outputId":"b3550e97-dfab-45b8-8878-90fa7457cc99","trusted":false},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"zhL52tmBvIwl","outputId":"2fdc1979-b513-4279-9da6-2100e156825e","trusted":false},"cell_type":"code","source":"history = model.fit(\n    train_data_gen,\n    epochs=epochs,\n    validation_data=val_data_gen,\n    steps_per_epoch=(total_train * (1 - validation_split) // batch_size),\n    validation_steps=(total_train * validation_split // batch_size)\n)","execution_count":null,"outputs":[]},{"metadata":{"id":"KnOiJhKltEgI"},"cell_type":"markdown","source":"## Visualize training results","execution_count":null},{"metadata":{"id":"wHdL6U8DtJfA","outputId":"90af056c-7a2a-433a-f38a-28c7d9316540","trusted":false},"cell_type":"code","source":"acc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\n\nloss=history.history['loss']\nval_loss=history.history['val_loss']\n\nepochs_range = range(epochs)\n\nplt.figure(figsize=(8, 8))\nplt.subplot(1, 2, 1)\nplt.plot(epochs_range, acc, label='Training Accuracy')\nplt.plot(epochs_range, val_acc, label='Validation Accuracy')\nplt.legend(loc='lower right')\nplt.title('Training and Validation Accuracy')\n\nplt.subplot(1, 2, 2)\nplt.plot(epochs_range, loss, label='Training Loss')\nplt.plot(epochs_range, val_loss, label='Validation Loss')\nplt.legend(loc='upper right')\nplt.title('Training and Validation Loss')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"toHjy4Iajf3V"},"cell_type":"markdown","source":"## Predict","execution_count":null},{"metadata":{"id":"Sea_fIAsTyK-","outputId":"01b5380d-a86d-43ca-9639-38d7b82df603","trusted":false},"cell_type":"code","source":"test_image_generator = ImageDataGenerator(rescale=1./255)\ntest_data_gen = test_image_generator.flow_from_directory(directory='./test/',\n                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),\n                                                           shuffle=False,\n                                                           class_mode=None,\n                                                           batch_size=1)","execution_count":null,"outputs":[]},{"metadata":{"id":"u8WECmqMo6FO","outputId":"cfa03a2e-15f4-46e2-a0d3-fdb76b9baedf","trusted":false},"cell_type":"code","source":"test_data_gen.reset()\npredictions = model.predict(test_data_gen,\n                            steps=total_test,\n                            verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"id":"wov_KApsFOc7"},"cell_type":"markdown","source":"Classes are `{'cat': 0, 'dog': 1}`. That matches output format so we can just round predictions value.","execution_count":null},{"metadata":{"id":"-jVa8fZxzq_9","outputId":"3a1e3082-0418-4c34-bb9b-ae45854a7423","trusted":false},"cell_type":"code","source":"train_data_gen.class_indices","execution_count":null,"outputs":[]},{"metadata":{"id":"Coq0Zd8Qjj3L"},"cell_type":"markdown","source":"## Check accuracy by comparing test files and corresponded predictions","execution_count":null},{"metadata":{"id":"P65S6iyZsPcm","outputId":"3e8e6806-a25e-4515-d042-b622531c17ed","trusted":false},"cell_type":"code","source":"test_path = pathlib.Path('./test/images')\npaths = [str(f) for f in test_path.glob('*')]\nfor path, pred in list(zip(paths, np.round(predictions)))[:10]:\n  plt.imshow(Image.open(path))\n  plt.show()\n  print(pred)","execution_count":null,"outputs":[]},{"metadata":{"id":"EEBvETUujpZb"},"cell_type":"markdown","source":"## Generate submittion.csv","execution_count":null},{"metadata":{"id":"S0hI7a9GgLM_","trusted":false},"cell_type":"code","source":"def extract_test_id(path):\n   file = path.split('/')[1]\n   return file.split('.')[0]\n\ntest_ids = [extract_test_id(f) for f in test_data_gen.filenames]","execution_count":null,"outputs":[]},{"metadata":{"id":"EwK8nrhlh9OH","trusted":false},"cell_type":"code","source":"zipped = zip(test_ids, np.round(predictions.flatten()))\nrecords = sorted(list(zipped), key=lambda kv: int(kv[0]))","execution_count":null,"outputs":[]},{"metadata":{"id":"R9LG8a5Li5ye","trusted":false},"cell_type":"code","source":"output = pd.DataFrame.from_records(records, columns=['id', 'label'])\noutput.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}