{"cells":[{"metadata":{"id":"u0rhrVMg6jdI"},"cell_type":"markdown","source":"# Dog Breed Identification\n\nWho's a good dog? Who likes ear scratches? Well, it seems those fancy deep neural networks don't have all the answers. However, maybe they can answer that ubiquitous question we all ask when meeting a four-legged stranger: what kind of good pup is that?\n\nIn this playground competition, you are provided a strictly canine subset of ImageNet in order to practice fine-grained image categorization. How well you can tell your Norfolk Terriers from your Norwich Terriers? With 120 breeds of dogs and a limited number training images per class, you might find the problem more, err, ruff than you anticipated.\n\n![Dataset samples](https://storage.googleapis.com/kaggle-competitions/kaggle/3333/media/border_collies.png)\n\n## Acknowledgments\n\nWe extend our gratitude to the creators of the [Stanford Dogs Dataset](http://vision.stanford.edu/aditya86/ImageNetDogs/) for making this competition possible: Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li.\n\n## Problem\n\nIdentifying the breed of as dog given an image of a dog.\n\n## Data\n\nThe data we're using is from Kaggle's [Dog Breed Identification](https://www.kaggle.com/c/dog-breed-identification/data) competition.\n\n# Evaluation\n\nThe evaluation is a file with prediction probabilities for each dog breed of each test image, as stated [here](https://www.kaggle.com/c/dog-breed-identification/overview/evaluation).\n\n## Features\n\nSome information about the data:\n\n* We're dealing with images (unstructured data) so it's probably best we use deep learning/transfer learning.\n* There are 120 breed of dogs (this means there are 120 different classes).\n* There are around 10,000+ images in the training set (these images have labels).\n* There are around 10,000+ images in the test set (theses images have no labels, because we'll want to predict them).\n\n## Workspace setup","execution_count":null},{"metadata":{"id":"EEm4Ndjj72MD","outputId":"aee68430-edd3-449e-9e7c-0ba9df3f8b2c","trusted":true},"cell_type":"code","source":"# Import necessary tools\nimport tensorflow as tf\nimport tensorflow_hub as hub\nimport pandas as pd\nimport numpy as np\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\n\nimport os\nimport datetime","execution_count":null,"outputs":[]},{"metadata":{"id":"QWxYwz959oAe","outputId":"d93e2a67-603f-43fa-f199-b9c32dc63265","trusted":true},"cell_type":"code","source":"print(\"TF version:\", tf.__version__)\nprint(\"TF Hub version:\", hub.__version__)\n\n# Check for GPU availability\nprint(\"GPU\", \"available\" if tf.config.list_physical_devices(\"GPU\") else \"not\")","execution_count":null,"outputs":[]},{"metadata":{"id":"FoYZZWnxDu7w","trusted":true},"cell_type":"code","source":"DATA_PATH = \"/kaggle/input/dog-breed-identification/\"\nMODELS_PATH = \"/kaggle/working/models/\"\nLOGS_PATH = \"/kaggle/working/logs/\"\nOUTPUT_PATH = \"/kaggle/working/output/\"\n\n# Make sure that the required directories path exists\nif not os.path.isdir(MODELS_PATH):\n    os.makedirs(MODELS_PATH)\nif not os.path.isdir(LOGS_PATH):\n    os.makedirs(LOGS_PATH)\nif not os.path.isdir(OUTPUT_PATH):\n    os.makedirs(OUTPUT_PATH)","execution_count":null,"outputs":[]},{"metadata":{"id":"kafTozhU-EDw"},"cell_type":"markdown","source":"# Data Loading\n\nAs with all machine learning models, our data has to be in numerical format. So that's what we'll be doing first: turning our images into **Tensors**\n\nLet's tart by accessing our data and checking out the labels.","execution_count":null},{"metadata":{"id":"QUM-_J91Dby-","outputId":"6f4b5fe2-42f4-482b-c86c-ed679342211c","trusted":true},"cell_type":"code","source":"labels_csv = pd.read_csv(DATA_PATH + \"labels.csv\")\ndisplay(labels_csv.describe())\ndisplay(labels_csv.head())","execution_count":null,"outputs":[]},{"metadata":{"id":"be9k8FO_EeMn","outputId":"279632cb-8446-4e31-a318-ea0a968168cf","trusted":true},"cell_type":"code","source":"# How manu images are there of each breed?\nlabels_csv.breed.value_counts().plot.bar(figsize=(20, 10))","execution_count":null,"outputs":[]},{"metadata":{"id":"ANxIeedEEzT0","outputId":"693843a0-3de1-4a40-cf56-8c27bb531782","trusted":true},"cell_type":"code","source":"labels_csv.breed.value_counts().median()","execution_count":null,"outputs":[]},{"metadata":{"id":"4IuUtp0QE6cv","outputId":"0d52decf-f45d-4896-d222-a04196dff427","trusted":true},"cell_type":"code","source":"# Let's view an image\nfrom IPython.display import Image\nImage(DATA_PATH + \"train/001513dfcb2ffafc82cccf4d8bbaba97.jpg\")","execution_count":null,"outputs":[]},{"metadata":{"id":"xVCvgSfmFIGz"},"cell_type":"markdown","source":"## Getting images and their labels\n\nLet's get a list of all our image file path names.","execution_count":null},{"metadata":{"id":"dC3paW7ZFfem","outputId":"85f7dc56-aa5e-42c5-81e1-dc18a0412322","trusted":true},"cell_type":"code","source":"filenames = [DATA_PATH + f\"train/{fname}.jpg\" for fname in labels_csv[\"id\"]]\nfilenames[:10]","execution_count":null,"outputs":[]},{"metadata":{"id":"WPGhDjk8Fjen","outputId":"9c1f2bf2-4a11-42aa-e143-ffd71e8ad2c5","trusted":true},"cell_type":"code","source":"# Check whether number of filenames matches number of actual image files\nif len(os.listdir(DATA_PATH + \"train\")) == len(filenames):\n    print(\"Filenames match actual amount of files! Proceed.\")\nelse:\n    print(\n        \"Filenames do not match actual amount of files! Check target directory.\"\n    )","execution_count":null,"outputs":[]},{"metadata":{"id":"Zn-PhqjEHWyL","outputId":"fd09ae0e-d83f-4b3f-faa2-7b1045854b12","trusted":true},"cell_type":"code","source":"# One more check\nprint(labels_csv.breed[9000])\nImage(filenames[9000])","execution_count":null,"outputs":[]},{"metadata":{"id":"DFa7tVdYHj6G"},"cell_type":"markdown","source":"Since we've got our training image filepaths in a list, let's prepare our labels.","execution_count":null},{"metadata":{"id":"HqY91o6ZH7NS","outputId":"fb1e87c0-acf2-45fc-9def-e5701daf8821","trusted":true},"cell_type":"code","source":"labels = labels_csv.breed.values\nlabels","execution_count":null,"outputs":[]},{"metadata":{"id":"ENNjFjccIKzF","outputId":"8a916927-fe69-4017-b32c-e2991689ab7e","trusted":true},"cell_type":"code","source":"len(labels)","execution_count":null,"outputs":[]},{"metadata":{"id":"t_7enDW9IMoe","outputId":"97a2f7ba-d1ce-45a5-a6d3-a5bf2335689e","trusted":true},"cell_type":"code","source":"# See if number of labels matches the number of filenames\nif len(labels) == len(filenames):\n    print(\"Number of labels matches number of filenames!\")\nelse:\n    print(\n        \"Number of labels does note matches number of filenames! Check data directory.\"\n    )","execution_count":null,"outputs":[]},{"metadata":{"id":"Ckxb-NdRIV1M","outputId":"60e1bbae-a2d5-43f9-b052-0f51b673f25f","trusted":true},"cell_type":"code","source":"# Find the uniques label values\nunique_breeds = np.unique(labels)\nprint(len(unique_breeds))\nprint(unique_breeds)","execution_count":null,"outputs":[]},{"metadata":{"id":"fII8JLl_LvSm"},"cell_type":"markdown","source":"## One-Hot Encoding","execution_count":null},{"metadata":{"id":"2pfhqe4GN_iH","outputId":"3f61d8dc-bf74-484d-80fc-9cac633b1d46","trusted":true},"cell_type":"code","source":"# Turn a single label into an array of booleans (one-hot array)\nprint(labels[0])\nlabels[0] == unique_breeds","execution_count":null,"outputs":[]},{"metadata":{"id":"20mSJR51OIP_","outputId":"14210177-7d2c-4c67-c7f2-b0955a7c5c5a","trusted":true},"cell_type":"code","source":"# Turn every label into a boolean array\none_hot_labels = [label == unique_breeds for label in labels]\none_hot_labels[:2]","execution_count":null,"outputs":[]},{"metadata":{"id":"j8g1g5gjOdyc"},"cell_type":"markdown","source":"## Creating our own validation set\n\nSince the dataset from Kaggle doesn't come with a validation set, we're going to create our own.","execution_count":null},{"metadata":{"id":"sZHHq_H2P-V3","trusted":true},"cell_type":"code","source":"# Setup X & y\nX = filenames\ny = one_hot_labels","execution_count":null,"outputs":[]},{"metadata":{"id":"PUlz4pRXQD6F"},"cell_type":"markdown","source":"We're going to start off experimenting with ~ 1000 images and increase as needed.","execution_count":null},{"metadata":{"id":"TESiysDUQLio","trusted":true},"cell_type":"code","source":"# Set number of images to use for experimenting\nNUM_IMAGES = 1000","execution_count":null,"outputs":[]},{"metadata":{"id":"Fz3z5rkbQRAE","outputId":"08e30b0e-26bd-42bc-b7dd-94af292ee460","trusted":true},"cell_type":"code","source":"# Split our data into training and validation of total size NUM_IMAGES\nX_train, X_val, y_train, y_val = train_test_split(X[:NUM_IMAGES],\n                                                  y[:NUM_IMAGES],\n                                                  test_size=0.2,\n                                                  random_state=42)\n\nlen(X_train), len(X_val), len(y_train), len(y_val)","execution_count":null,"outputs":[]},{"metadata":{"id":"DWpEM8ShSrqI","outputId":"0e58802f-2b36-4dc2-bf92-0571dddee83a","trusted":true},"cell_type":"code","source":"# Let's have a look on our training data\nX_train[:2], y_train[:2]","execution_count":null,"outputs":[]},{"metadata":{"id":"GY912IYsTNR2"},"cell_type":"markdown","source":"## Preprocessing Image (turning images into Tensors)\n\nTo preprocess our images into Tensors we're going to write a function which does a few things:\n\n1. Take a image filepath as input\n2. Use TensorFlow to read the file and save it to a variable `image`\n3. Turn our `image` (a jpg) into Tensors\n4. Resize the `image` to be a shape of (224, 224)\n5. return the modified `image`\n","execution_count":null},{"metadata":{"id":"m3OuD5ToUu2s","trusted":true},"cell_type":"code","source":"IMG_SIZE = 224\n\n\n# Function for preprocessing images\ndef process_image(image_path, img_size=IMG_SIZE):\n    \"\"\"\n  Takes an image filepath and turns it into a Tensor\n  \"\"\"\n    # Read the image file\n    image = tf.io.read_file(image_path)\n    # Turn the jpeg image into numerical Tensor with 3 color channels (Red, Green, Blue)\n    image = tf.image.decode_jpeg(image, channels=3)\n    # Convert the color channels values range from 0-255 to 0-1\n    image = tf.image.convert_image_dtype(image, tf.float32)\n    # Resize the image to our desired values (224, 224)\n    image = tf.image.resize(image, size=(img_size, img_size))\n    # Return the modified image\n    return image","execution_count":null,"outputs":[]},{"metadata":{"id":"TdP3f8LOWNRL"},"cell_type":"markdown","source":"## Turning our data into batches\n\nWhy turn our data into batches?\n\nLet's say you're trying to process 10,000+ images in one go... they all might not fit into memory.\n\nSo that's why we do about 32 (this is batch size) images at a time (you can manually adjust the batch size if needed).\n\nIn order to use TensorFlow effectively, we need our data in the form of Tensor tuples which look like this: (`image`, `label`)","execution_count":null},{"metadata":{"id":"xMBdIMUvW1Lr","trusted":true},"cell_type":"code","source":"# Simple function to return a tuple (image, label)\ndef get_image_label(image_path, label):\n    \"\"\"\n  Takes an image filepath name and the associated label, processes the image and return a tuple of (image, label)\n  \"\"\"\n    image = process_image(image_path)\n    return image, label","execution_count":null,"outputs":[]},{"metadata":{"id":"wcQWDBn5XYqX"},"cell_type":"markdown","source":"Now we've got a way to turn our data into tuples of Tensors in the form: (`image`, `label`), let's make a function to turn all of our data (`X` & `y`) into batches!","execution_count":null},{"metadata":{"id":"WddWFlqVXnqj","trusted":true},"cell_type":"code","source":"# Define the batch size. 32 is a good start\nBATCH_SIZE = 32\n\n\n# Function to turn data into batches\ndef create_data_batches(X,\n                        y=None,\n                        batch_size=BATCH_SIZE,\n                        valid_data=False,\n                        test_data=False):\n    \"\"\"\n  Creates batches of data out of image (X) and label (y) pairs. Shuffles the data if it's validation data.\n  Also accepts test data as input (no labels).\n  \"\"\"\n    # If the data is test dataset, we probably don't have labels\n    if test_data:\n        print(\"Creating test data batches...\")\n        data = tf.data.Dataset.from_tensor_slices(\n            (tf.constant(X)))  # only filepaths (no labels)\n        data_batch = data.map(process_image).batch(BATCH_SIZE)\n        return data_batch\n\n    # If the data is a valid dataset, we don't need to shuffle ir\n    elif valid_data:\n        print(\"Creating validation data batches...\")\n        data = tf.data.Dataset.from_tensor_slices((\n            tf.constant(X),  # filepaths\n            tf.constant(y)))  # labels\n        data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n        return data_batch\n\n    else:\n        print(\"Creating training data batches...\")\n        data = tf.data.Dataset.from_tensor_slices(\n            (tf.constant(X), tf.constant(y)))\n        # Shuffling pathnames and labels bafore mapping image processor function is faster than shuffling images\n        data = data.shuffle(buffer_size=len(X))\n\n        # Create (image, label) tuples (this also turns the image path into a preprocessed image)\n        data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n\n        return data_batch","execution_count":null,"outputs":[]},{"metadata":{"id":"I14pj7mFfqAM","outputId":"6ab09591-f33c-4ab4-9302-7b1df11a816d","trusted":true},"cell_type":"code","source":"# Create training and validation data batches\ntrain_data = create_data_batches(X_train, y_train)\nval_data = create_data_batches(X_val, y_val, valid_data=True)","execution_count":null,"outputs":[]},{"metadata":{"id":"Ar1wIBURf3uG","outputId":"866217d6-551c-40d5-eb9d-489424e545d6","trusted":true},"cell_type":"code","source":"# Check out the different attributes of our data batches\ntrain_data.element_spec, val_data.element_spec","execution_count":null,"outputs":[]},{"metadata":{"id":"kwwjh7OBgB_c"},"cell_type":"markdown","source":"## Visualizing data batches\n\nOur data is now in batches. However, these can be a little hard to understand/comprehend. Let's visualize them!","execution_count":null},{"metadata":{"id":"sw8O3sOhxZhH","trusted":true},"cell_type":"code","source":"# Function for viewing images ina a data batch\ndef show_25_images(images, labels):\n    \"\"\"\n  Displays a plot of a 25 of images and their labels from a data batch.\n  \"\"\"\n    # Setup the figure\n    plt.figure(figsize=(10, 10))\n    # Loop through the 25 * for displaying 25 images:\n    for i in range(25):\n        ax = plt.subplot(5, 5, i + 1)\n        # Display an image\n        plt.imshow(images[i])\n        # Add the image label as the title\n        plt.title(unique_breeds[labels[i].argmax()])\n        # Turn the grid lines off\n        plt.axis(\"off\")","execution_count":null,"outputs":[]},{"metadata":{"id":"As2w1yZ1ysJe","outputId":"614d8cda-1856-474f-9810-1aba43029358","trusted":true},"cell_type":"code","source":"# Let's visualize our training set\ntrain_images, train_labels = next(train_data.as_numpy_iterator())\nshow_25_images(train_images, train_labels)","execution_count":null,"outputs":[]},{"metadata":{"id":"G6tPomE3y4ai","outputId":"2b3108ff-e3cd-44ee-c347-1edd78ecf827","trusted":true},"cell_type":"code","source":"# Now let's visualize our validation set\nval_images, val_labels = next(val_data.as_numpy_iterator())\nshow_25_images(val_images, val_labels)","execution_count":null,"outputs":[]},{"metadata":{"id":"Kx7kvZtV1ASj"},"cell_type":"markdown","source":"# Building a model\n\nBefore we build a model, there are a few things we need to define:\n\n* The input shape (our images shape, in the form of Tensors) to our model.\n* The output shape (image label, in the form of Tensors) of our model.\n* The URL of the model we want to use from [TensorFlow hub]( https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4)","execution_count":null},{"metadata":{"id":"pPqR3nbp1neG","trusted":true},"cell_type":"code","source":"INPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE,\n               3]  # batch, hieght, width, color channels\n\n# Setup output shape of our model\nOUTPUT_SHAPE = len(unique_breeds)\n\n# Setup the MobileNetV2 model URL from TensorFlow hub\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4\"","execution_count":null,"outputs":[]},{"metadata":{"id":"rJHOID0M3UXA"},"cell_type":"markdown","source":"Now we've got our inputs, outputs and model ready to g, let's put them together into a Keras deep learning model.\n\nKnowing this, let's create a function which:\n* Takes a input shape, output shape and the model we've chosen as parameters.\n* Defines the layers in a Keras model in sequential fashion (do this first, then this, then that).\n* Compiles the mode l(says it should be evaluated and improved).\n* Builds the model (tells the model the input shape it'll be getting).\n* Return the model\n\nAll these steps can be found [here](https://www.tensorflow.org/guide/keras/sequential_model)","execution_count":null},{"metadata":{"id":"_ffhOYKp38P4","trusted":true},"cell_type":"code","source":"# Function which builds a Keras model\ndef create_model(input_shape=INPUT_SHAPE,\n                 output_shape=OUTPUT_SHAPE,\n                 model_url=MODEL_URL):\n    print(\"Building model with:\", model_url)\n\n    # Setup the model layers\n    model = tf.keras.Sequential([\n        hub.KerasLayer(model_url),  # layer 1 (input layer)\n        tf.keras.layers.Dense(units=output_shape,\n                              activation=\"softmax\")  # layer 2 (output layer)\n    ])\n\n    # Compile the model\n    model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),\n                  optimizer=tf.keras.optimizers.Adam(),\n                  metrics=[\"accuracy\"])\n\n    # Build the model\n    model.build(input_shape)\n\n    return model","execution_count":null,"outputs":[]},{"metadata":{"id":"JpoJtgmt5Oi6","outputId":"6d50343b-54a8-4323-b88a-d13054750bc5","trusted":true},"cell_type":"code","source":"model = create_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"LcZYf49O5vz1"},"cell_type":"markdown","source":"## Creating callbacks\n\nCallbacks are helper functions a model can use during training to do such things as save its prograss, check its progress or stop training early if a model stops improving.\n\nWe'll create two callbacks. One for TehsorBoard, which helps track our model progress, and another for early stopping, which prevents our model from training for too long.","execution_count":null},{"metadata":{"id":"uzsX9h1_-WLm"},"cell_type":"markdown","source":"### TensorBoard callback\n\nTo setup a TensorBoard callback, we need to do 3 things:\n1. Load the TensorBoard notebook extension\n2. Create a TensorBoard callback which is able to save logs to a directory and pass it to our model's `fit()` function.\n3. visualize our models training log with the `%tensorboard` magic function (we'll do this after model training)","execution_count":null},{"metadata":{"id":"hqE5sQzk8Zlb","trusted":true},"cell_type":"code","source":"# Load TensorBoard notebook extension\n%load_ext tensorboard","execution_count":null,"outputs":[]},{"metadata":{"id":"9DOPbCEN8fNc","trusted":true},"cell_type":"code","source":"# Function to build a TensorBoard callback\ndef create_tensorboard_callback():\n    # Create a log directory for storing TensorBoard logs\n    logdir = os.path.join(\n        LOGS_PATH,  # make it so the logs get tracked whenever we run an experiment\n        datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n    return tf.keras.callbacks.TensorBoard(logdir)","execution_count":null,"outputs":[]},{"metadata":{"id":"I7Xcnl-o9tXg"},"cell_type":"markdown","source":"### Early Stopping callback\n\n[Early Stopping](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping) helps stop our model overfitting by stopping training if a certain metric stops improving.","execution_count":null},{"metadata":{"id":"oYi2oeVr-REn","trusted":true},"cell_type":"code","source":"# Create Early Stopping callback\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\",\n                                                  patience=3)","execution_count":null,"outputs":[]},{"metadata":{"id":"LtGBVQcO-nTn"},"cell_type":"markdown","source":"# Training a model (on a subset of data)\n\nOur first model is only going to train on 1000 images, to make sure everything is working.\n\nLet's create a function which trains a model.\n* Create a model using `create_model()`.\n* Setup a TensorBoard callback using `create_tensorboard_callback()`.\n* Call the `fit()` function on our model passing it the training data, validation data, number of epochs to train for (`NUM_EPOCHS`) and the callbacks we'd like to use.\n* Return the model.","execution_count":null},{"metadata":{"id":"HoD2iGhe-2PE","trusted":true},"cell_type":"code","source":"NUM_EPOCHS = 100\n\n\n# Function to train and return a trained model\ndef train_model(num_epochs=NUM_EPOCHS):\n    \"\"\"\n  Trains a given model and return the trained version.\n  \"\"\"\n    # Create a model\n    model = create_model()\n\n    # Create a new TensorBoard session everytime we train a model\n    tensorboard = create_tensorboard_callback()\n\n    # Fit the model to the data passing it the callbacks we created\n    model.fit(x=train_data,\n              epochs=NUM_EPOCHS,\n              validation_data=val_data,\n              validation_freq=1,\n              callbacks=[tensorboard, early_stopping])\n\n    # Return the fitted model\n    return model","execution_count":null,"outputs":[]},{"metadata":{"id":"sc6lZCKfAN0A","outputId":"69c3ecf8-5feb-421e-e8bc-70fcb94d269d","trusted":true},"cell_type":"code","source":"# Fit the model to the data\nmodel = train_model()","execution_count":null,"outputs":[]},{"metadata":{"id":"XX_n54J1AaZb"},"cell_type":"markdown","source":"## Checking the TensorBoard logs\n\nThe TensorBoard maginc function (`%tensorboard`) will access the logs directory we created earlier and visualize its contents","execution_count":null},{"metadata":{"id":"b5U5H549As_O","outputId":"4f1a20f1-f1e3-422b-ee61-3ca94f66fb09","trusted":true},"cell_type":"code","source":"%tensorboard --logdir $LOGS_PATH","execution_count":null,"outputs":[]},{"metadata":{"id":"8zMdqgZbA2Wl"},"cell_type":"markdown","source":"# Making and evaluating prediction using a pre-trained model","execution_count":null},{"metadata":{"id":"imw7aZ-_A8Z2","outputId":"853d5ee3-1f2d-4488-f58c-e899d248f1f3","trusted":true},"cell_type":"code","source":"predictions = model.predict(val_data, verbose=1)\npredictions","execution_count":null,"outputs":[]},{"metadata":{"id":"TDaOR8FWQL7S","outputId":"6a4ae36e-ffbd-41a8-d70e-c67f64c30d1d","trusted":true},"cell_type":"code","source":"index = 42\nprint(predictions[index])\nprint(f\"Max value (probability of prediction): {np.max(predictions[index])}\")\nprint(f\"Sum: {np.sum(predictions[index])}\")\nprint(f\"Max index: {np.argmax(predictions[index])}\")\nprint(f\"Predicted label: {unique_breeds[np.argmax(predictions[index])]}\")","execution_count":null,"outputs":[]},{"metadata":{"id":"ELj8wnliQ4LU","outputId":"06d8e03c-10b6-4ccc-9aa6-68ca8989f045","trusted":true},"cell_type":"code","source":"unique_breeds[113]","execution_count":null,"outputs":[]},{"metadata":{"id":"hiSnkzNxRBbd"},"cell_type":"markdown","source":"Having the above functionality is greate but we want to be able to do it at scale. And it would be even better if we could see the image the prediction is being made on!\n\n**Note:** Prediction probabilities are also know as *confidence levels*","execution_count":null},{"metadata":{"id":"GLzkbPdaTuQG","trusted":true},"cell_type":"code","source":"# Turn prediction probabilities into their respective label (easier to understand)\ndef get_pred_label(prediction_probabilities):\n    \"\"\"\n  Turn an array of prediction probabilities into a label\n  \"\"\"\n    return unique_breeds[np.argmax(prediction_probabilities)]","execution_count":null,"outputs":[]},{"metadata":{"id":"85RsSSLfUDjh","outputId":"2c1a4613-5845-4c9b-f7f4-d46ccef1aaa2","trusted":true},"cell_type":"code","source":"# Get a predicted label based on an array of prediction probabilities\nget_pred_label(predictions[81])","execution_count":null,"outputs":[]},{"metadata":{"id":"C3DCkYXDURoN"},"cell_type":"markdown","source":"Now sice our validation data is still in a batch dataset, we'll have to unbatchify it to make prediction on validation images and them compare those predictions to the validation labels (truth labels)","execution_count":null},{"metadata":{"id":"eE4TLucWUhBD","trusted":true},"cell_type":"code","source":"# Function to unbatchify a batch dataset\ndef unbatchify(data):\n    \"\"\"\n  Takes a batched dataset of (image, label) Tensors and return separate arrays\n  of images and labels\n  \"\"\"\n    images = []\n    labels = []\n    # Loop trhough unbatched data\n    for image, label in data.unbatch().as_numpy_iterator():\n        images.append(image)\n        labels.append(get_pred_label(label))\n\n    return images, labels","execution_count":null,"outputs":[]},{"metadata":{"id":"2ePmKQmsVIzU","outputId":"ed811d3d-2459-472e-ece0-e4af986f8cc4","trusted":true},"cell_type":"code","source":"# Unbatchify the validation data\nval_images, val_labels = unbatchify(val_data)\nval_images[0], val_labels[0]","execution_count":null,"outputs":[]},{"metadata":{"id":"qqH1pMGLVSUN"},"cell_type":"markdown","source":"Now we've got ways to get:\n* Prediction labels\n* validation labels\n* validation images\n\nLet's make some function to make these all a bit more visualize.\n\nWe'll create a function which:\n* Takens an array of prediction probabilities, an array of truth labels, an array of images and an integer.\n* Convert the prediction probabilities to a predicted label.\n* Plot the predicted label, its probability, the truth and the target image in a single plot.","execution_count":null},{"metadata":{"id":"AASRtqv7WP4Q","trusted":true},"cell_type":"code","source":"def plot_pred(prediction_probabilities, labels, images, n=1):\n    \"\"\"\n  View the prediction ground truth and image for sample n\n  \"\"\"\n    pred_prob, true_label, image = prediction_probabilities[n], labels[\n        n], images[n]\n\n    # Get the pred label\n    pred_label = get_pred_label(pred_prob)\n\n    # Plot the image & remove ticks\n    plt.imshow(image)\n    plt.xticks([])\n    plt.yticks([])\n\n    # Change the color of the title depending if the prediction is right or wrong\n    if pred_label == true_label:\n        color = \"green\"\n    else:\n        color = \"red\"\n\n    # Change plot title to be predicted, probability of prediction and truth label\n    plt.title(f\"{pred_label} {np.max(pred_prob)*100:2.0f}% {true_label}\",\n              color=color)","execution_count":null,"outputs":[]},{"metadata":{"id":"nc0Q9KCiXOhI","outputId":"0beee9ae-d713-400a-f260-c40fcff37df7","trusted":true},"cell_type":"code","source":"plot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images)","execution_count":null,"outputs":[]},{"metadata":{"id":"ENqw6HFGXW-L"},"cell_type":"markdown","source":"Now we've got one function to visualize our models top predictions, let's make another to view our models top 10 predictions\n\nThis function will:\n* Take an input of prediction probabilities array and a ground truth array and an integer.\n* Find the prediction using `get_pred_label()`\n* Find the top 10:\n  * Prediction probabilities indexes\n  * Prediction probabilities values\n  *Prediction labels\n* Plot the top 10 prediction probability values and labels, coloring the true label green.","execution_count":null},{"metadata":{"id":"NeYSnwPlYF4k","trusted":true},"cell_type":"code","source":"def plot_pred_conf(prediction_probabilities, labels, n=1):\n    \"\"\"\n  Plot the top 10 highest prediction confidences along with the truth label for\n  sample n\n  \"\"\"\n    pred_prob, true_label = prediction_probabilities[n], labels[n]\n\n    # Get the predicted label\n    pred_label = get_pred_label(pred_prob)\n\n    # Find the top 10 prediction confidence indexes\n    top_10_pred_indexes = pred_prob.argsort()[-10:][::-1]\n    # Find the top 10 prediction connfidence values\n    top_10_pred_values = pred_prob[top_10_pred_indexes]\n    # Find the top 10 prediction labels\n    top_10_pred_labels = unique_breeds[top_10_pred_indexes]\n\n    # Setup plot\n    top_plot = plt.bar(np.arange(len(top_10_pred_labels)),\n                       top_10_pred_values,\n                       color=\"grey\")\n    plt.xticks(np.arange(len(top_10_pred_labels)),\n               labels=top_10_pred_labels,\n               rotation=\"vertical\")\n\n    # Change color of true label\n    if np.isin(true_label, top_10_pred_labels):\n        top_plot[np.argmax(\n            top_10_pred_labels == true_label)].set_color(\"green\")\n    else:\n        pass","execution_count":null,"outputs":[]},{"metadata":{"id":"G2Oqys9mJrj1","outputId":"4d6405c9-b7e8-4462-bbe5-33eaf43a2ea2","trusted":true},"cell_type":"code","source":"plot_pred_conf(prediction_probabilities=predictions, labels=val_labels, n=9)","execution_count":null,"outputs":[]},{"metadata":{"id":"XwZcvoxQJ7X8"},"cell_type":"markdown","source":"Now we've got some function to help us visualize our predictions and evaluate our model, let's check out a few.","execution_count":null},{"metadata":{"id":"3zgJNyixKYfP","outputId":"70bde653-8fb6-4999-9d88-d5a1a0cb44e1","trusted":true},"cell_type":"code","source":"# Let's check out a few predictions and their different values\ni_multiplier = 10\nn_rows = 3\nn_cols = 2\nn_images = n_cols * n_rows\nplt.figure(figsize=(10 * n_cols, 5 * n_rows))\nfor i in range(n_images):\n    plt.subplot(n_rows, 2 * n_cols, 2 * i + 1)\n    plot_pred(prediction_probabilities=predictions,\n              labels=val_labels,\n              images=val_images,\n              n=i + i_multiplier)\n    plt.subplot(n_rows, 2 * n_cols, 2 * i + 2)\n    plot_pred_conf(prediction_probabilities=predictions,\n                   labels=val_labels,\n                   n=i + i_multiplier)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"iFgrxykNxkAu"},"cell_type":"markdown","source":"## Confusion matrix","execution_count":null},{"metadata":{"id":"RBmtRpzHLOTQ","trusted":true},"cell_type":"code","source":"def plot_conf_matrix(prediction_probabilities, labels):\n    \"\"\"\n  Plot the confusion matrix of a trained model given its prediction\n  probabilities and desired labels\n  \"\"\"\n    # First, we get the corresponding labels of the predictions\n    pred_labels = [\n        get_pred_label(pred_probs) for pred_probs in prediction_probabilities\n    ]\n\n    # Check which breeds are present either in true and predicted labels\n    breeds_in_true_labels = set(labels)\n    breeds_in_pred_labels = set(pred_labels)\n    breeds_in_set = [\n        breed for breed in unique_breeds\n        if breed in breeds_in_pred_labels and breed in breeds_in_true_labels\n    ]\n\n    # Computes the confusion matrix\n    conf_mat = confusion_matrix(labels, pred_labels, labels=breeds_in_set)\n\n    # Builds the confusion matrix dataframe (for the x and y ticks in the heatmap)\n    conf_df = pd.DataFrame(conf_mat,\n                           index=breeds_in_set,\n                           columns=breeds_in_set)\n    conf_df.dropna(inplace=True)\n\n    # Now we plot the confusion matrix\n    fig, ax = plt.subplots(figsize=(20, 20))\n    conf_plot = sns.heatmap(conf_df, annot=True, cbar=False)\n\n    plt.title(\"Confusion matrix\")\n    plt.xlabel(\"True label\")\n    plt.ylabel(\"Predicted label\")","execution_count":null,"outputs":[]},{"metadata":{"id":"JTAXYY-6ka_R","outputId":"ff1428cc-c01e-41fe-ca39-8fab360495a0","trusted":true},"cell_type":"code","source":"plot_conf_matrix(predictions, val_labels)","execution_count":null,"outputs":[]},{"metadata":{"id":"uoUASJyDxwZe"},"cell_type":"markdown","source":"## Saving and reloading a trained model","execution_count":null},{"metadata":{"id":"35bisxuOx0Rq","trusted":true},"cell_type":"code","source":"# Create a function to save a model\ndef save_model(model, suffix=None):\n    \"\"\"\n  Save a given model in a model directory and appends a suffix (string)\n  \"\"\"\n    # Create a model directory with current time\n    modeldir = os.path.join(MODELS_PATH,\n                            datetime.datetime.now().strftime(\"%Y%m%d_%H%M%S\"))\n    model_path = modeldir + \"_\" + suffix + \".h5\"  # model save format\n    print(f\"Saving model to: {model_path}...\")\n    model.save(model_path)\n    return model_path","execution_count":null,"outputs":[]},{"metadata":{"id":"Y6Qfw7RAyq9J","trusted":true},"cell_type":"code","source":"# Create a function to load a trained model\ndef load_model(model_path):\n    print(f\"Loading saved model from: {model_path}...\")\n    model = tf.keras.models.load_model(\n        model_path, custom_objects={\"KerasLayer\": hub.KerasLayer})\n    return model","execution_count":null,"outputs":[]},{"metadata":{"id":"YSCH7O85y_lQ"},"cell_type":"markdown","source":"Now we've got a function to save and load a trained mode, let's make sure they work!","execution_count":null},{"metadata":{"id":"_kETYBnTzE38","outputId":"e85f4530-af0a-43c7-8f14-51aea6526538","trusted":true},"cell_type":"code","source":"# Save our model trained on 1000 images\nmodel_path = save_model(model, suffix=\"1000_images_mobilenetv2_Adam\")","execution_count":null,"outputs":[]},{"metadata":{"id":"3G3ARlrIzN9J","outputId":"210940c1-1a8c-47f5-c067-afb919483e1f","trusted":true},"cell_type":"code","source":"# Load a trained model\nloaded_1000_image_model = load_model(model_path)","execution_count":null,"outputs":[]},{"metadata":{"id":"H61puoab0RQn","outputId":"042bb48f-66e3-41f6-8498-790843f5edc4","trusted":true},"cell_type":"code","source":"model.evaluate(val_data, )","execution_count":null,"outputs":[]},{"metadata":{"id":"BroILhtX0WV-","outputId":"a0bfa5aa-fb93-437b-d2b0-3832ae93a9fb","trusted":true},"cell_type":"code","source":"model.metrics_names","execution_count":null,"outputs":[]},{"metadata":{"id":"377EamQqM3O1"},"cell_type":"markdown","source":"# Training on the full data","execution_count":null},{"metadata":{"id":"rlkCM7n7NXMI","outputId":"c9827095-faf2-41fc-9fe9-09453afe396e","trusted":true},"cell_type":"code","source":"# Create a data batch with the full data set\nfull_data = create_data_batches(X, y)","execution_count":null,"outputs":[]},{"metadata":{"id":"n13-SZZTNheR","outputId":"a7d5963c-a0d9-4db4-aa96-54d0e5aabc33","trusted":true},"cell_type":"code","source":"full_data","execution_count":null,"outputs":[]},{"metadata":{"id":"uUYupfHiNii4","outputId":"f502bde7-a6eb-4017-c1a0-e3a5f6993d8a","trusted":true},"cell_type":"code","source":"# Create a model for full model\nfull_model = create_model()","execution_count":null,"outputs":[]},{"metadata":{"id":"jVCjfYgIPGro","trusted":true},"cell_type":"code","source":"# Create full model callbacks\nfull_model_tensorboard = create_tensorboard_callback()\n# No validation set when training on all the data, so we can't monitor validation accuracy\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(\n    monitor=\"accuracy\", patience=3)","execution_count":null,"outputs":[]},{"metadata":{"id":"JeYKVw5pPoma","outputId":"56194ba5-7028-4a0a-c8bc-a930ed2d3fb7","trusted":true},"cell_type":"code","source":"# Fit the full model to the full data\nfull_model.fit(x=full_data,\n               epochs=NUM_EPOCHS,\n               callbacks=[full_model_tensorboard, full_model_early_stopping])","execution_count":null,"outputs":[]},{"metadata":{"id":"xHn21NdFP3fQ","outputId":"09af6f0c-bb2b-442c-ad1b-9cfdc6e04018","trusted":true},"cell_type":"code","source":"full_model_path = save_model(full_model, suffix=\"full_image_set_mobilenetv2_Adam\")","execution_count":null,"outputs":[]},{"metadata":{"id":"ee0S7Rb8QAAe","outputId":"d3b326fb-e4ee-4e2d-920e-365ee52c2932","trusted":true},"cell_type":"code","source":"loaded_full_model = load_model(full_model_path)","execution_count":null,"outputs":[]},{"metadata":{"id":"BIQIpmmGqFdQ"},"cell_type":"markdown","source":"# Making predictions on the test dataset\n\nSince our model has been trained on images in the form of Tensor batches, to make predictions on the test data, we'll have to get it into the same format.\n\nTo make predictions on the test data, we'll:\n\n* Get the test images filenames.\n* Convert the filenames into test data batches using `create_data_batches()` and setting the `test_data` parameter to `True` (since the test data doesn't have labels).\n* Make predictions arrar by passing the test batches tot the `predict()` method called on our model.","execution_count":null},{"metadata":{"id":"VcZwRhuprIfA","outputId":"8487cfa0-5aa9-43db-c403-a3a32530a412","trusted":true},"cell_type":"code","source":"# Load test image filenames\ntest_path = DATA_PATH + \"test/\"\ntest_filenames = [test_path + fname for fname in os.listdir(test_path)]\ntest_filenames[:10]","execution_count":null,"outputs":[]},{"metadata":{"id":"sFB_TiMrrXTx","outputId":"c6e9468c-b0f4-4bf6-861e-af9be8f00ba6","trusted":true},"cell_type":"code","source":"len(test_filenames)","execution_count":null,"outputs":[]},{"metadata":{"id":"UZWRVqqkr8d8","outputId":"74a2684b-08d1-4ffd-b57d-900ec45349d7","trusted":true},"cell_type":"code","source":"# Create test data batch\ntest_data = create_data_batches(test_filenames, test_data=True)","execution_count":null,"outputs":[]},{"metadata":{"id":"oPi6MuZpsCp1","outputId":"e941c961-0134-4011-9ddc-0b31f57e48f3","trusted":true},"cell_type":"code","source":"test_data","execution_count":null,"outputs":[]},{"metadata":{"id":"z7fV1fPysHNY","outputId":"bdd75f1f-8d8e-4f78-fb67-d0821c2b31b4","trusted":true},"cell_type":"code","source":"# Make predictions on test data batch using the loaded full model\ntest_predictions = loaded_full_model.predict(test_data, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"id":"RyxtGsrLsRSo","trusted":true},"cell_type":"code","source":"# Save predictions (NumPy arrary) to csv file (for access later)\nnp.savetxt(OUTPUT_PATH + \"preds_array.csv\", test_predictions, delimiter=\",\")","execution_count":null,"outputs":[]},{"metadata":{"id":"zQ8bme4LA4DX","trusted":true},"cell_type":"code","source":"test_predictions = np.loadtxt(OUTPUT_PATH + \"preds_array.csv\", delimiter=\",\")","execution_count":null,"outputs":[]},{"metadata":{"id":"J8fQAuaPBBCm","outputId":"b462e645-b647-4675-e172-64491dd488cb","trusted":true},"cell_type":"code","source":"test_predictions","execution_count":null,"outputs":[]},{"metadata":{"id":"ak8n4xNPBEvO","outputId":"f71278bb-e410-454e-9f18-9c922806ccf4","trusted":true},"cell_type":"code","source":"test_predictions.shape","execution_count":null,"outputs":[]},{"metadata":{"id":"QbswbOQSBG81"},"cell_type":"markdown","source":"# Preparing test dataset predictions for Kaggle\n\nLooking ar the [Kaggle sample submission](https://www.kaggle.com/c/dog-breed-identification/overview/evaluation), we find that it wants our models prediction probability outputs in a DataFrame with and ID and a column for each dog breed.\n\nTo get the data in this format, we'll:\n* Create a pandas DataFrame with and ID column as well as a column for each dog breed.\n* Add data to the ID column by extracting the test image ID's from their filepaths.\n* Add data ( the prediction probabilities) to each of the dog breed columns.\n* Export the DataFrame as a CSV to submit it to Kaggle.","execution_count":null},{"metadata":{"id":"9MDO-LHgPA6o","outputId":"bca5baca-fbd8-48b9-bc0c-7fac30e9c21f","trusted":true},"cell_type":"code","source":"# Create a pandas DataFrame with empty columns\npreds_df = pd.DataFrame(columns=[\"id\"] + list(unique_breeds))\npreds_df","execution_count":null,"outputs":[]},{"metadata":{"id":"NzLmxlDkPQAw","outputId":"c3d2b352-97c3-41c1-8f28-301d91c8a1a0","trusted":true},"cell_type":"code","source":"# Append test image ID's to predictions DataFrame\ntest_ids = [os.path.splitext(path)[0] for path in os.listdir(test_path)]\npreds_df[\"id\"] = test_ids\npreds_df.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"B2MMnPVsPihJ","outputId":"0a83f464-bd06-4556-9e84-61c6f2af4898","trusted":true},"cell_type":"code","source":"# Add the prediction probabilities to each dog breed column\npreds_df[list(unique_breeds)] = test_predictions\npreds_df.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"LTF-ZbxZP4_5","trusted":true},"cell_type":"code","source":"# Save our predictions dataframe to CSV for submission to Kaggle\npreds_df.to_csv(OUTPUT_PATH +\n                \"full_model_predictions_submission_1_mobilenetV2.csv\",\n                index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}