{"cells":[{"metadata":{"id":"E2c8QEUzlZHu","colab_type":"text"},"cell_type":"markdown","source":"
Dogs vs. Cats, Keras CNN with transfer learning (Inception V3)

\n
The structure of this notebook:\n
    \n
  1. Create a model by transfer learning\n
  2. Load the dataset from kaggle\n
  3. Split the dataset into training set and validation set\n
  4. See sample images\n
  5. Data Augmentation\n
  6. Training\n
  7. Prediction\n
\n
\nReference:
Thie code of this notebook is inspired by TensorFlow in Practice Specialization.
The theoretical background is explained in Deep Learning Specialization. The lecuture videos are also available on youtube. Both are offered by deeplearning.ai"},{"metadata":{"id":"s5TXvatoNuIT","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Open this notebook on google colab or kaggle kernel\n# Change runtime type to GPU\n# In December 2019, tensorflow 2.0 is only available in google colab or kaggle kernel\n# ! pip install tensorflow==2.0.0","execution_count":null,"outputs":[]},{"metadata":{"id":"lbFmQdsZs5eW","colab_type":"code","outputId":"0550a268-ffb3-455b-dd5b-d27f9e2469a8","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# Import the relevant libraries\nimport os\nimport json\nimport zipfile\nimport shutil\nimport random\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\n\nfrom tensorflow.keras import layers\nfrom tensorflow.keras import Model\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.preprocessing.image import load_img\nfrom shutil import copyfile\n\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{"id":"3LcLza17rIzX","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"FGAoiSniOMVj","colab_type":"text"},"cell_type":"markdown","source":"
Step 1. Create a model by transfer learning

\nTransfer learning is a technique to use parameters of a pre-trained model. In image classification, there are many models pre-trained on ImageNet. ImageNet is a database of images which has more than 14 million images with 20,000 categories. Several models including ResNet50 and InceptionV3 are available in keras.
\nFor theortical background of CNN(convolutional neural network) and transfer learninng, deeplearning.ai lecture videos by professor Andrew Ng are very helpful.\nPlease find lecture #1-20 in the link . "},{"metadata":{"colab_type":"code","id":"1xJZ5glPPCRz","outputId":"14f1f2ce-6e0b-4bdc-e884-6747b51309e5","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# Import the inception V3 model \nfrom tensorflow.keras.applications.inception_v3 import InceptionV3\n\n# Clear any exiting model\ntf.keras.backend.clear_session()\n\npre_trained_model = InceptionV3(input_shape = (150,150,3), # reshape images to 150 by 150 by 3 channels\n include_top = False, # get straight to the CNN layer\n weights = 'imagenet') # use the builed-in weight pre-trained on imagent\n\n# Make all the layers in the pre-trained model non-trainable/frozen\nfor layer in pre_trained_model.layers:\n layer.trainable = False","execution_count":null,"outputs":[]},{"metadata":{"id":"RNZxB3F2IAGg","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Print the InceptionV3 model summary\n# pre_trained_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"CFsUlwdfs_wg","colab_type":"code","outputId":"06a5a2ec-cc03-4fb5-af2d-b64b70477eed","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":35}},"cell_type":"code","source":"# Pick one layer from the miffde of the model\n# We well build new dense layers after this layer\n\nlast_layer = pre_trained_model.get_layer('mixed7')\nprint('last layer output shape: ', last_layer.output_shape)\nlast_output = last_layer.output","execution_count":null,"outputs":[]},{"metadata":{"id":"-bsWZWp5oMq9","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Define a Callback class that stops training once accuracy reaches a threshold\n\nclass myCallback(tf.keras.callbacks.Callback):\n def on_epoch_end(self, epoch, logs={}):\n if(logs.get('acc')>0.875):\n print(\"\\nReached 87.5% accuracy so cancelling training!\")\n self.model.stop_training = True","execution_count":null,"outputs":[]},{"metadata":{"colab_type":"code","id":"BMXb913pbvFg","outputId":"c9ed51ed-f665-4281-ed8b-f5613cb49cd5","trusted":true,"scrolled":true,"colab":{"base_uri":"https://localhost:8080/","height":1000}},"cell_type":"code","source":"from tensorflow.keras.optimizers import RMSprop\n\n# Add layers after the picked last_layer\n# Note that the layers below are trainable. \n# On the other hand, the layers up to the last_output is fronzen and non-trainable.\n\n# Flatten the output layer to 1 dimension\nx = layers.Flatten()(last_output)\n# Add a fully connected layer with 1,024 hidden units and ReLU activation\nx = layers.Dense(1024, activation='relu')(x)\n# Add a dropout rate of 0.2\nx = layers.Dropout(0.2)(x) \n# Add a final sigmoid layer for binary classification\nx = layers.Dense(1, activation='sigmoid')(x) \n\n# instantiate a new model \nmodel = Model(pre_trained_model.input ,x) \n\n# Compile a model\nmodel.compile(optimizer = RMSprop(lr=0.0001), # We use RMSprop in this case\n loss = 'binary_crossentropy', # cats vs. dogs is a binary problem\n metrics = ['acc']) # 'acc' stands for accuracy\n\n# See the summary of the model \n# model.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"jsCbyhjkXH2D","colab_type":"text"},"cell_type":"markdown","source":"Droptout: please find lecture #6 in the link .
\nRMSProp: lecture #21 in the link ."},{"metadata":{"id":"uuAj0zBKI29z","colab_type":"text"},"cell_type":"markdown","source":"So far, the model has learned nothing from the Dogs vs. Cats dataset. \n
"},{"metadata":{"id":"rIxaKT37lw08","colab_type":"text"},"cell_type":"markdown","source":"
Step2.1 Load the dataset from kaggle

\nIf you are in the kaggle kernel implement the followings:"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(os.listdir(\"../input/dogs-vs-cats\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Delete the directory we will create if it already exists\ntry:\n shutil.rmtree(\"/tmp/cat_or_dog/\") \nexcept:\n pass\n\n# Create a new directory for the dataset\ntry:\n os.mkdir(\"/tmp/cat_or_dog/\")\nexcept:\n pass","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Unzip　the training data\nlocal_zip = '../input/dogs-vs-cats/train.zip'\nzip_ref = zipfile.ZipFile(local_zip, 'r')\nzip_ref.extractall('/tmp/cat_or_dog')\nzip_ref.close()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"os.listdir('/tmp/cat_or_dog')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
If you are inside the kaggle kernel, Skip Step2.2 and jump tp Step 2.3

\n>
"},{"metadata":{},"cell_type":"markdown","source":"
Step2.2 Load the dataset from kaggle

\nIf yur are o[](http://)utside the kaggle kernel, implement the following:"},{"metadata":{"id":"Gm_FipiBg7FM","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Login to kaggle \n# Join the competition where the data are stored\n# Go to My account and click \"Create new API Token\" \n# A json file will be downloaded. It can be opened in any text file\n# Set your kaggle username and key in the dictionary below \n\n\n# api_token = {\"username\":\"your_kaggle_username\",\"key\":\"your_kaggle_key\"}","execution_count":null,"outputs":[]},{"metadata":{"id":"N54O81EpjPhY","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Create a .kaggle　directory in the root directory \n\n\n# try:\n# os.mkdir(\"/root/.kaggle/\")\n# except:\n# pass","execution_count":null,"outputs":[]},{"metadata":{"id":"FNxeh-lKjTgR","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Create a json file which stores the api keys\n\n\n# with open('/root/.kaggle/kaggle.json', 'w') as file:\n# json.dump(api_token, file)","execution_count":null,"outputs":[]},{"metadata":{"id":"7CHXI_n3kOdq","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Set owner permission to the json file \n\n# ! chmod 600 /root/.kaggle/kaggle.json","execution_count":null,"outputs":[]},{"metadata":{"id":"Ov5jtfy8i-J5","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Delete the directory we will create if it already exists\n\n# try:\n# shutil.rmtree(\"/tmp/cat_or_dog/\") \n# except:\n# pass\n\n# # Create a new directory for the dataset\n# try:\n# os.mkdir(\"/tmp/cat_or_dog/\")\n# except:\n# pass","execution_count":null,"outputs":[]},{"metadata":{"id":"2YwYdmGYkK82","colab_type":"code","outputId":"775df341-0eb2-4df2-9d2f-afed766b07ea","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":195}},"cell_type":"code","source":"# Go to the data tab in the kaggle competition and copy the command \n# After -p is the directory to store the data\n\n# ! kaggle competitions download -c dogs-vs-cats -p /tmp/cat_or_dog/","execution_count":null,"outputs":[]},{"metadata":{"id":"Ixs821_vlOJ6","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Unzip　the training data\n\n\n# local_zip = '/tmp/cat_or_dog/train.zip'\n# zip_ref = zipfile.ZipFile(local_zip, 'r')\n# zip_ref.extractall('/tmp/cat_or_dog')\n# zip_ref.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
Step2.3 Load the dataset from kaggle

\nThe code below are the same for users outside the kaggle or inside the kaggle kernel"},{"metadata":{"id":"ITrsaV0Lm5du","colab_type":"code","outputId":"afbe8f32-d29c-4b1e-f97c-5509bf5fccd4","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# Cats and dogs images are stored in the same dirctory\nprint(os.listdir('/tmp/cat_or_dog'))\nprint(os.listdir('/tmp/cat_or_dog/train')[:10])\nfiles_list = os.listdir('/tmp/cat_or_dog/train')","execution_count":null,"outputs":[]},{"metadata":{"id":"Syz3b2oqoIZm","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Create directories for training and validation\n\nbase_train_dir = '/tmp/cat_or_dog/training/'\nbase_val_dir = '/tmp/cat_or_dog/validation/'\n\n# Delete the directory we will create if it already exists\ntry:\n shutil.rmtree(base_train_dir) \nexcept:\n pass\n\ntry:\n shutil.rmtree(base_val_dir) \nexcept:\n pass\n\n# Directory with our training cat/dog pictures\ntrain_cats_dir = os.path.join(base_train_dir, 'cats/')\ntrain_dogs_dir = os.path.join(base_train_dir, 'dogs/')\n\n# Directory with our validation cat/dog pictures\nval_cats_dir = os.path.join(base_val_dir, 'cats/')\nval_dogs_dir = os.path.join(base_val_dir, 'dogs/')\n\n\ntry:\n os.mkdir(base_train_dir)\n os.mkdir(base_val_dir)\n os.mkdir(train_cats_dir)\n os.mkdir(train_dogs_dir)\n os.mkdir(val_cats_dir)\n os.mkdir(val_dogs_dir) \nexcept:\n pass","execution_count":null,"outputs":[]},{"metadata":{"id":"l7ZgrDj1p8la","colab_type":"code","outputId":"e61b90bc-8294-4402-cd88-06b97c1b77b8","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# When using .flow_from_directory in the generator, the \"training\" directory should have sub-directories for the classes, \"dogs\" and \"cats\"\n# Another option is to use .flow_from_dataframe. In this case, a dataframe with filename and corresponding categories is required. \nprint(os.listdir(base_train_dir))\n\n# Same for the \"validation\" directory \nprint(os.listdir(base_val_dir))","execution_count":null,"outputs":[]},{"metadata":{"id":"TpeU2-a2mMrM","colab_type":"code","outputId":"b22ba12f-6e21-46ee-c4b6-73a47455d615","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":70}},"cell_type":"code","source":"# Shuffle the files before splting into training set and test set\nrandom.seed(0)\nrandom_files_list = random.sample(files_list,len(files_list))\nprint(random_files_list[:10])\nprint(len(files_list))\nprint(len(random_files_list))","execution_count":null,"outputs":[]},{"metadata":{"id":"ItT29Sr06w9l","colab_type":"code","outputId":"f34161b3-b71e-4a06-bdde-44ad64fc73be","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# Deleat empty files\n# In this dataset, there is no empty files\nSOURCE = '/tmp/cat_or_dog/train/'\n\nrandom_files_clean_list = []\n\nfor filename in random_files_list:\n file = os.path.join(SOURCE + filename)\n if os.path.getsize(file) > 0:\n random_files_clean_list.append(filename)\n else:\n print(filename + \" is zero length, so ignoring.\")\n\n# print(random_files_clean_list)\nprint(len(random_files_list))\nprint(len(random_files_clean_list))","execution_count":null,"outputs":[]},{"metadata":{"id":"adsPheSxME8U","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"iOBJ58uSL9rm","colab_type":"text"},"cell_type":"markdown","source":"
Step 3. Split the dataset into training set and validation set

\nFor details about training set and validation set, please find Andrew Ng's lecture #1 in the link, or #5 in the link."},{"metadata":{"id":"JtelVtkvq5sT","colab_type":"code","outputId":"39f83ac6-ad24-41d5-a579-afb419c40ac4","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":70}},"cell_type":"code","source":"# Split the files into training set and validation set\n\n# Set the ratio for validation set\nval_ratio = 0.3 \n\ntraining_list = random_files_clean_list[:int(len(random_files_clean_list)*(1 - val_ratio))]\nval_list = random_files_clean_list[-int(len(random_files_clean_list)*(val_ratio)):]\n\nprint(len(training_list))\nprint(len(val_list))\nprint(len(training_list) + len(val_list))","execution_count":null,"outputs":[]},{"metadata":{"id":"QVF6Nzt9zVpH","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Create the training set\n\nSOURCE = '/tmp/cat_or_dog/train/' # the current directory where files are stored\n\nfor filename in training_list:\n this_file = os.path.join(SOURCE + filename)\n cat_destination = os.path.join(train_cats_dir + filename)\n dog_destination = os.path.join(train_dogs_dir + filename)\n if 'cat' in filename:\n copyfile(this_file, cat_destination)\n elif 'dog' in filename:\n copyfile(this_file, dog_destination)","execution_count":null,"outputs":[]},{"metadata":{"id":"CoT6ioPs0xnV","colab_type":"code","outputId":"3c61f545-ab61-419d-8b87-5711749aeeda","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":88}},"cell_type":"code","source":"train_dog_fnames = os.listdir(train_dogs_dir)\ntrain_cat_fnames = os.listdir(train_cats_dir)\n\n\nprint(os.listdir(train_cats_dir)[:5])\nprint(os.listdir(train_dogs_dir)[:5])\nprint(len(os.listdir(train_cats_dir)))\nprint(len(os.listdir(train_dogs_dir)))","execution_count":null,"outputs":[]},{"metadata":{"id":"mYm2w_c73_LH","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Create the validation set\n\nSOURCE = '/tmp/cat_or_dog/train/' # the current directory where files are stored\n\nfor filename in val_list:\n this_file = os.path.join(SOURCE + filename)\n cat_destination = os.path.join(val_cats_dir + filename)\n dog_destination = os.path.join(val_dogs_dir + filename)\n\n \n if 'cat' in filename:\n copyfile(this_file, cat_destination)\n elif 'dog' in filename:\n copyfile(this_file, dog_destination)","execution_count":null,"outputs":[]},{"metadata":{"id":"Ba-mIUJk5ar3","colab_type":"code","outputId":"58262859-df92-442b-cbc2-b3ec402f4a90","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":52}},"cell_type":"code","source":"# print(os.listdir(val_cats_dir))\n# print(os.listdir(val_dogs_dir))\nprint(len(os.listdir(val_cats_dir)))\nprint(len(os.listdir(val_dogs_dir)))","execution_count":null,"outputs":[]},{"metadata":{"id":"P-9ZUmbgQdTy","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"sxlWZ1-xCUvc","colab_type":"text"},"cell_type":"markdown","source":"
Step 4. See sample images

"},{"metadata":{"id":"TplBVf3r-GPM","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"%matplotlib inline\n\nimport matplotlib.image as mpimg\nimport matplotlib.pyplot as plt\n\n# Parameters for our graph; we'll output images in a 4x4 configuration\nnrows = 4\nncols = 4\n\npic_index = 0 # Index for iterating over images","execution_count":null,"outputs":[]},{"metadata":{"id":"hXVYhXM--IUh","colab_type":"code","outputId":"43948624-4dbc-45a0-afd9-34a5083c0c33","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":670}},"cell_type":"code","source":"# Set up matplotlib fig, and size it to fit 4x4 pics\n# The 8 images on the upper half are dogs, others are cats \nfig = plt.gcf()\nfig.set_size_inches(ncols*3, nrows*3)\n\npic_index+=8\n\nnext_cat_pix = [os.path.join(train_cats_dir, fname) \n for fname in train_cat_fnames[ pic_index-8:pic_index] \n ]\n\nnext_dog_pix = [os.path.join(train_dogs_dir, fname) \n for fname in train_dog_fnames[ pic_index-8:pic_index]\n ]\n\nfor i, img_path in enumerate(next_cat_pix+next_dog_pix):\n # Set up subplot; subplot indices start at 1\n sp = plt.subplot(nrows, ncols, i + 1)\n sp.axis('Off') # don't show axes (or gridlines)\n\n img = mpimg.imread(img_path)\n plt.imshow(img)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"YhoG00pAR_uX","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"iYjSQ8SkRipI","colab_type":"text"},"cell_type":"markdown","source":"
Step 5. Data Augmentation

\nData augmentation is a technique to artifically create training data. It rotates, shifts, zooms, shears, or horizontally flips the training images. In keras ImageDataGenerator class, the original images are unchaged. It just implements data augmentation when they are feeded into the training/fitting.\nFor details, please find Andrew Ng's lecture #21 in the link ."},{"metadata":{"colab_type":"code","id":"O4s8HckqGlnb","outputId":"92143d7a-d75e-4711-8e1a-c95c31092c2b","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":50}},"cell_type":"code","source":"# Add our data-augmentation parameters to ImageDataGenerator\ntrain_datagen = ImageDataGenerator(rescale = 1./255., # rescale the images within 0-1. This yield better results when handling images. \n rotation_range = 40,\n width_shift_range = 0.2,\n height_shift_range = 0.2,\n shear_range = 0.2,\n zoom_range = 0.2,\n horizontal_flip = True)\n\n# Note that the validation data should not be augmented\ntest_datagen = ImageDataGenerator(rescale = 1.0 / 255.0)\n\n# Flow training images in batches of 20 using train_datagen generator\ntrain_generator = train_datagen.flow_from_directory(base_train_dir,\n batch_size = 20,\n class_mode = 'binary',\n target_size = (150, 150)) \n\n# Flow validation images in batches of 20 using test_datagen generator\nvalidation_generator = test_datagen.flow_from_directory(base_val_dir,\n batch_size = 20,\n class_mode = 'binary',\n target_size = (150,150))","execution_count":null,"outputs":[]},{"metadata":{"id":"xIqlAVJlQjXt","colab_type":"code","outputId":"c42732a9-273f-4610-af43-8cf639be832b","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":33}},"cell_type":"code","source":"# See the indices of the classes\n# In keras ImageDataGenerator class, classes are indexed in alphabetical order\ntrain_generator.class_indices","execution_count":null,"outputs":[]},{"metadata":{"id":"7m5BudBmVPI5","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"CQGC2HSsVLGo","colab_type":"text"},"cell_type":"markdown","source":"
Step 6. Training

\nWhen implementing training, it is very important to understand the concept of bias/underfitting and variance/overfitting. When a model has high bias, it has low performance both on the training set and the validation set. If the model has high variance, the model performs very well on the training set but has low performance on the validation set.

When the model is underfitting, there are two solutions among others. One way is to do more training or train on more epochs. If it still not work well, we have to change the model architecture. For example, if the target objects are not in the center of the images, neural networks without convolutional layers tend to have poor performance (high bias). In that case, CNN is a better choice.

\nWhen the model is overfitting, there are muliple choices. Regularization/add penalties, drop out, or training on fewer epochs are among them. Data augmentation is also a way to reduce overfitting.

\nBias and Variance: Please find Andrew Ng's lecture #2 in the link.
\n"},{"metadata":{"colab_type":"code","id":"Blhq2MAUeyGA","outputId":"9b9c7efe-9fce-4ee5-cfdb-4a03528e37ff","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":650}},"cell_type":"code","source":"# Fit the model the training data\n# Take the log of the training in history\ncallbacks = myCallback()\nhistory = model.fit_generator(\n generator = train_generator, # feed the training data via the generator\n steps_per_epoch = 8, # this is the batch size. parameters are updated per this batch size\n epochs = 100, # number of cycles. In one epoch, the whole dataset is used once.\n verbose = 2, # print out the logs\n callbacks = [callbacks], # use callbacks we set before\n validation_data = validation_generator, # feed the validation data via the generator\n validation_steps = 50 \n)","execution_count":null,"outputs":[]},{"metadata":{"id":"C2Fp6Se9rKuL","colab_type":"code","outputId":"b943633d-4640-4db2-e6a3-3a4aac12c6be","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":297}},"cell_type":"code","source":"# Plot the accuracy history \nimport matplotlib.pyplot as plt\nacc = history.history['acc']\nval_acc = history.history['val_acc']\nloss = history.history['loss']\nval_loss = history.history['val_loss']\n\nepochs = range(len(acc))\n\nplt.plot(epochs, acc, 'r', label='Training accuracy')\nplt.plot(epochs, val_acc, 'b', label='Validation accuracy')\nplt.title('Training and validation accuracy')\nplt.legend(loc=0)\nplt.figure()\n\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"LLr7W2WDfvD_","colab_type":"text"},"cell_type":"markdown","source":"
"},{"metadata":{"id":"4Ccn5sPhZfJn","colab_type":"text"},"cell_type":"markdown","source":"
Step 7. Prediction

\nPrediction of the unlabeled test set images is the final stage. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Inside the kaggle kernel\n# Unzip　the test data\nlocal_zip = '../input/dogs-vs-cats/test1.zip'\nzip_ref = zipfile.ZipFile(local_zip, 'r')\nzip_ref.extractall('/tmp/cat_or_dog')\nzip_ref.close()","execution_count":null,"outputs":[]},{"metadata":{"id":"hPFYEjhXIY3d","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Outside the kaggle kernel\n# Unzip　the test data\n# local_zip = '/tmp/cat_or_dog/test1.zip'\n# zip_ref = zipfile.ZipFile(local_zip, 'r')\n# zip_ref.extractall('/tmp/cat_or_dog')\n# zip_ref.close()","execution_count":null,"outputs":[]},{"metadata":{"id":"LT_6zTqaGiM1","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"base_test_dir = '/tmp/cat_or_dog/test1/'\n\n# print(os.listdir(base_test_dir))","execution_count":null,"outputs":[]},{"metadata":{"id":"-K_8m9ZqxgMN","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# To use .flow_from_dataframe. in the generator, we have to create a dataframe \ntest_filenames = os.listdir(base_test_dir)\ntest_df = pd.DataFrame({\n 'filename': test_filenames\n})\nnb_samples = test_df.shape[0]","execution_count":null,"outputs":[]},{"metadata":{"id":"Fd7gcgj6xqYp","colab_type":"code","outputId":"ab0bff98-0f78-4b07-ac3d-58378998f578","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":33}},"cell_type":"code","source":"# Create a generator for test set\ntest_gen = ImageDataGenerator(rescale=1./255.0)\nbatch_size=20\n\ntest_generator = test_gen.flow_from_dataframe(\n test_df, \n base_test_dir, \n x_col='filename',\n y_col=None,\n class_mode=None,\n batch_size=batch_size,\n target_size=(150, 150),\n shuffle=False\n)","execution_count":null,"outputs":[]},{"metadata":{"id":"CiMW1lojyFAc","colab_type":"code","trusted":true,"colab":{}},"cell_type":"code","source":"# Predict the test set\npredict = model.predict_generator(test_generator, steps=np.ceil(nb_samples/batch_size))\nthreshold = 0.5\ntest_df['category'] = np.where(predict > threshold, 1,0) # we can also ","execution_count":null,"outputs":[]},{"metadata":{"id":"Kke1JJQziZc8","colab_type":"code","outputId":"4e14eb81-b0e3-42ed-c675-600b8946d160","trusted":true,"colab":{"base_uri":"https://localhost:8080/","height":1000}},"cell_type":"code","source":"# See sample results, # 1 = dog, 0 = cat\nsample_test = test_df.head(18)\nsample_test.head()\nplt.figure(figsize=(9, 18))\nfor index, row in sample_test.iterrows():\n filename = row['filename']\n category = row['category']\n img = load_img(base_test_dir+filename, target_size=(150,150))\n plt.subplot(6, 3, index+1)\n plt.imshow(img)\n plt.xlabel(filename + '(' + \"{}\".format(category) + ')' ) \nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"Z3RiPRuTjsKW","colab_type":"text"},"cell_type":"markdown","source":"
Submit (from outside the kaggle)

\n
\nNote that we cannot submit past competiton."},{"metadata":{"id":"KxLRMtyuyjkz","colab_type":"code","trusted":false,"colab":{}},"cell_type":"code","source":"submission_df = test_df.copy()\nsubmission_df['id'] = submission_df['filename'].str.split('.').str[0]\nsubmission_df['label'] = submission_df['category']\nsubmission_df.drop(['filename', 'category'], axis=1, inplace=True)\nsubmission_df.to_csv('/tmp/cat_or_dog/submission_20191230.csv', index=False)","execution_count":0,"outputs":[]},{"metadata":{"id":"bKW2_qF60BZz","colab_type":"code","outputId":"43419435-a49a-4027-daa7-21b3492c0a04","trusted":false,"colab":{"base_uri":"https://localhost:8080/","height":33}},"cell_type":"code","source":"! kaggle competitions submit -c dogs-vs-cats -f /tmp/cat_or_dog/ submission_20191229.csv -m \"First submission\"","execution_count":0,"outputs":[]}],"metadata":{"colab":{"name":"dogs_cats_keras_CNN_transfer_learning_inceptionv3_v2.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"accelerator":"GPU","language_info":{"mimetype":"text/x-python","nbconvert_exporter":"python","name":"python","file_extension":".py","version":"3.5.4","pygments_lexer":"ipython3","codemirror_mode":{"version":3,"name":"ipython"}}},"nbformat":4,"nbformat_minor":1}