{"cells":[{"metadata":{},"cell_type":"markdown","source":"# IMPORTING OUR LIBRARIES AND MODULES"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Essential Libraries for data analysis and manipulation\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom pandas_profiling import ProfileReport\n\n# Deep Learning libraries\nimport tensorflow as tf\nimport tensorflow_hub as hub\n\n# Image Processing and Visualization\nfrom IPython.display import display, Image\nfrom matplotlib.pyplot import imread\n\n# Data Spliting Module\nfrom sklearn.model_selection import train_test_split\n\n# Other\nimport os\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Checking that we run TensorFlow v2 and the GPU is working \nprint(\"TF version:\", tf.__version__)\nprint(\"GPU\", \"available (YESS!!!!)\" if tf.config.list_physical_devices(\"GPU\") else \"not available :(\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Loading Data "},{"metadata":{},"cell_type":"markdown","source":"we are provided with a training set and a test set of images of dogs. Each image has a filename that is its unique id. The dataset comprises 120 breeds of dogs. \n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Loading the labels of our data\nlabels_csv = pd.read_csv(\"../input/dog-breed-identification/labels.csv\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Lets do fast EDA on our Labels "},{"metadata":{"trusted":true},"cell_type":"code","source":"Labels_Profile = ProfileReport(labels_csv, progress_bar= False)\nLabels_Profile.to_notebook_iframe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"raw","source":"It turns out that our labels are very clean (do not contain missed values), ID is unique (no duplicates), and we have 120 unique dog breeds."},{"metadata":{},"cell_type":"markdown","source":"Lets visualize our classes(breeds) to check for imbalance "},{"metadata":{"trusted":true},"cell_type":"code","source":"\nlabels_csv[\"breed\"].value_counts().plot.bar(figsize=(20, 10));","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It seems that we have no significant imbalance between classes."},{"metadata":{},"cell_type":"markdown","source":"### Getting images and their labels\n\nSince we've got the image ID's and their labels in a DataFrame (`labels_csv`), we'll use it to create:\n* A list a filepaths to training images\n* An array of all labels\n* An array of all unique labels\n\nWe'll only create a list of filepaths to images rather than importing them all to begin with. This is because working with filepaths (strings) is much efficient than working with images."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create pathnames from image ID's\nfilenames = [\"../input/dog-breed-identification/train/\" + fname + \".jpg\" for fname in labels_csv[\"id\"]]\n\n# Check the first 5 filenames to see if output of our feature engineering function\nfilenames[:5]\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check whether number of filenames matches number of actual image files\nif len(os.listdir(\"../input/dog-breed-identification/train/\")) == len(filenames):\n  print(\"Filenames match actual amount of files!\")\nelse:\n  print(\"Filenames do not match actual amount of files, check the target directory.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If everything worked, we should see a match up.\n\nLet's do one more check. Visualizing directly from a filepath."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check an image directly from a filepath\nImage(filenames[42])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nNow we've got our image filepaths together, let's get the labels.\n\nWe'll take them from `labels_csv` and turn them into a NumPy array for handling them with numpy methods as needed"},{"metadata":{"trusted":true},"cell_type":"code","source":"# convert labels column to NumPy array\nlabels = labels_csv[\"breed\"].to_numpy() \n#check the first 10 elements in the label ndarray\nlabels[:10]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"now lets do the same thing as before, compare the amount of labels to number of filenames to assert that everything is working properly before moving to the next stage!"},{"metadata":{"trusted":true},"cell_type":"code","source":"# See if number of labels matches the number of filenames\nif len(labels) == len(filenames):\n  print(\"Number of labels matches number of filenames!\")\nelse:\n  print(\"Number of labels does not match number of filenames, check data directories.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"If it all worked, we should have the same amount of images and labels.\n\nFinally, since a machine learning model can't take strings as input (what `labels` currently is), we'll have to convert our labels to numbers. \n\nTo begin with, we'll find all of the unique dog breed names.\n\nThen we'll go through the list of `labels` and compare them to unique breeds and create a list of booleans indicating which one is the real label (`True`) and which ones aren't (`False`)."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Find the unique label values\nunique_breeds = np.unique(labels)\nlen(unique_breeds)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Turn every label into a boolean array\nboolean_labels = [label == np.array(unique_breeds) for label in labels]\nboolean_labels[:2]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we've got our labels in a numeric format and our image filepaths easily accessible (they aren't numeric yet), let's split our data up."},{"metadata":{},"cell_type":"markdown","source":"### Creating our own validation set\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Setup X & y variables\nX = filenames\ny = boolean_labels","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since we're working with 10,000+ images, it's a good idea to work with a portion of them to for fast expermintation.\nLet's start experimenting with 1000 and increase it as we need."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set number of images to use for experimenting\nNUM_IMAGES = 1000 \n\n# Split them into training and validation using NUM_IMAGES \nX_train, X_val, y_train, y_val = train_test_split(X[:NUM_IMAGES],\n                                                  y[:NUM_IMAGES], \n                                                  test_size=0.2,\n                                                  random_state=42)\n\nlen(X_train), len(y_train), len(X_val), len(y_val)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preprocessing images (turning images into Tensors)\n"},{"metadata":{},"cell_type":"markdown","source":"\n\nTo preprocess our images into Tensors we're going to write a function which does a few things:\n1. Takes an image filename as input.\n2. Uses TensorFlow to read the file and save it to a variable, `image`.\n3. Turn our `image` (a jpeg file) into Tensors.\n4. Resize the `image` to be of shape (224, 224).\n5. Return the modified `image`.\n\nA good place to read about this type of function is the [TensorFlow documentation on loading images](https://www.tensorflow.org/tutorials/load_data/images). \n\nwe resize our image into (224, 224) because this is the size of input our pre-trained model (we'll see this soon from TesnoFlow hub) takes, an image which is (224, 224, 3).\n\n3 's the number of colour channels per pixel, red, green and blue.\n\n"},{"metadata":{},"cell_type":"markdown","source":"**Ok, now let's build that function we were talking about.**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define image size\nIMG_SIZE = 224\n\ndef process_image(image_path):\n  \"\"\"\n  Takes an image file path and turns it into a Tensor.\n  \"\"\"\n  # Read in image file\n  image = tf.io.read_file(image_path)\n  # Turn the jpeg image into numerical Tensor with 3 colour channels (Red, Green, Blue)\n  image = tf.image.decode_jpeg(image, channels=3)\n  # Convert the colour channel values from 0-225 values to 0-1 values\n  image = tf.image.convert_image_dtype(image, tf.float32)\n  # Resize the image to our desired size (224, 244)\n  image = tf.image.resize(image, size=[IMG_SIZE, IMG_SIZE])\n  return image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating data batches\nTo accelerate computing using GPU and to ingest our data into the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a simple function to return a tuple (image, label)\ndef get_image_label(image_path, label):\n  \"\"\"\n  Takes an image file path name and the associated label,\n  processes the image and returns a tuple of (image, label).\n  \"\"\"\n  image = process_image(image_path)\n  return image, label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the batch size, 32 is a good default\nBATCH_SIZE = 32\n\n# Create a function to turn data into batches\ndef create_data_batches(x, y=None, batch_size=BATCH_SIZE, valid_data=False, test_data=False):\n  \"\"\"\n  Creates batches of data out of image (x) and label (y) pairs.\n  Shuffles the data if it's training data but doesn't shuffle it if it's validation data.\n  Also accepts test data as input (no labels).\n  \"\"\"\n  # If the data is a test dataset, we probably don't have labels\n  if test_data:\n    print(\"Creating test data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x))) # only filepaths\n    data_batch = data.map(process_image).batch(BATCH_SIZE)\n    return data_batch\n  \n  # If the data if a valid dataset, we don't need to shuffle it\n  elif valid_data:\n    print(\"Creating validation data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x), # filepaths\n                                               tf.constant(y))) # labels\n    data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n    return data_batch\n\n  else:\n    # If the data is a training dataset, we shuffle it\n    print(\"Creating training data batches...\")\n    # Turn filepaths and labels into Tensors\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x), # filepaths\n                                              tf.constant(y))) # labels\n    \n    # Shuffling pathnames and labels before mapping image processor function is faster than shuffling images\n    data = data.shuffle(buffer_size=len(x))\n\n    # Create (image, label) tuples (this also turns the image path into a preprocessed image)\n    data = data.map(get_image_label)\n\n    # Turn the data into batches\n    data_batch = data.batch(BATCH_SIZE)\n  return data_batch","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create training and validation data batches\ntrain_data = create_data_batches(X_train, y_train)\nval_data = create_data_batches(X_val, y_val, valid_data=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check out the different attributes of our data batches\ntrain_data.element_spec, val_data.element_spec","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"our data in batches, more specifically, they're in Tensor pairs of (images, labels) ready for use on a GPU.\n\nBut having our data in batches can be a bit of a hard concept to understand. Let's build a function which helps us visualize what's going on under the hood.\n\n"},{"metadata":{},"cell_type":"markdown","source":"### Visualizing data batches"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a function for viewing images in a data batch\ndef show_25_images(images, labels):\n  \"\"\"\n  Displays 25 images from a data batch.\n  \"\"\"\n  # Setup the figure\n  plt.figure(figsize=(10, 10))\n  # Loop through 25 (for displaying 25 images)\n  for i in range(25):\n    # Create subplots (5 rows, 5 columns)\n    ax = plt.subplot(5, 5, i+1)\n    # Display an image\n    plt.imshow(images[i])\n    # Add the image label as the title\n    plt.title(unique_breeds[labels[i].argmax()])\n    # Turn gird lines off\n    plt.axis(\"off\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Looping over our Data batch in oreder to view our images**"},{"metadata":{},"cell_type":"markdown","source":"To make computation efficient, a batch is a tighly wound collection of Tensors.\n\nSo to view data in a batch, we've got to unwind it.\n\nWe can do so by calling the [`as_numpy_iterator()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#as_numpy_iterator) method on a data batch.\n\nThis will turn our a data batch into something which can be iterated over.\n\nPassing an iterable to [`next()`](https://docs.python.org/3/library/functions.html#next) will return the next item in the iterator.\n\nIn our case, next will return a batch of 32 images and label pairs.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Visualize training images from the training data batch\ntrain_images, train_labels = next(train_data.as_numpy_iterator())\nshow_25_images(train_images, train_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creating and training a model\n\nNow our data is ready, let's prepare it modelling. We'll use a pretrained model from [TensorFlow Hub](https://tfhub.dev/)(**transfer learning**).\n\n#### Why use a pretrained model?\n\nBuilding a machine learning model and training it on lots from scratch can be expensive and time consuming.\n\nTransfer learning helps eliviate some of these by taking what another model has learned and using that information with your own problem.\n\n### Building a model\n\nBefore we build a model, there are a few things we need to define:\n* The input shape (images, in the form of Tensors) to our model.\n* The output shape (image labels, in the form of Tensors) of our model.\n* The URL of the model we want to use."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Setup input shape according to the pretrained-model input shape\nINPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE, 3] # batch, height, width, colour channel ass\n\n# Setup output shape of the model\nOUTPUT_SHAPE = len(unique_breeds) # number of unique labels\n\n# Setup model URL from TensorFlow Hub\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we've got the inputs, outputs and model we're using ready to go. We can start to put them together\n\nThere are many ways of building a model in TensorFlow but one of the best ways to get started is to [use the Keras API](https://www.tensorflow.org/guide/keras/overview).\n\nDefining a deep learning model in Keras can be as straightforward as saying, \"here are the layers of the model, the input shape and the output shape, let's go!\"\n\nKnowing this, let's create a function which:\n* Takes the input shape, output shape and the model we've chosen's URL as parameters.\n* Defines the layers in a Keras model in a sequential style.\n* Compiles the model (says how it should be evaluated and improved).\n* Builds the model (tells it what kind of input shape it'll be getting).\n* Returns the model.\n\n"},{"metadata":{},"cell_type":"markdown","source":"## Building a model using Keras & TF hub APIs"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a function which builds a Keras model\ndef create_model(input_shape=INPUT_SHAPE, output_shape=OUTPUT_SHAPE, model_url=MODEL_URL):\n  print(\"Building model with:\", MODEL_URL)\n\n  # Setup the model layers using sequential mode.\n  model = tf.keras.Sequential([\n    hub.KerasLayer(MODEL_URL), # Layer 1 (the pre-trained model )\n    tf.keras.layers.Dense(units=OUTPUT_SHAPE, \n                          activation=\"softmax\") # Layer 2 (output layer)\n  ])\n\n  # Compile the model\n  model.compile(\n      loss=tf.keras.losses.CategoricalCrossentropy(), \n      optimizer=tf.keras.optimizers.SGD(), \n      metrics=[\"accuracy\"] \n  )\n\n  # Build the model\n  model.build(INPUT_SHAPE) \n  \n  return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a model and check its details\nmodel = create_model()\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Creating callbacks\n\nThe two callbacks we're going to add are a TensorBoard callback and an Early Stopping callback.\n\n#### TensorBoard Callback\n\n[TensorBoard](https://www.tensorflow.org/tensorboard/get_started) helps provide a visual way to monitor the progress of your model during and after training.\n\nIt can be used [directly in a notebook](https://www.tensorflow.org/tensorboard/tensorboard_in_notebooks) to track the performance measures of a model such as loss and accuracy.\n\nTo set up a TensorBoard callback and view TensorBoard in a notebook, we need to do three things:\n1. Load the TensorBoard notebook extension.\n2. Create a TensorBoard callback which is able to save logs to a directory and pass it to our model's `fit()` function.\n3. Visualize the our models training logs using the `%tensorboard` magic function (we'll do this later on)."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Load the TensorBoard notebook extension\n%load_ext tensorboard\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import datetime\n\n# Create a function to build a TensorBoard callback\ndef create_tensorboard_callback():\n  # Create a log directory for storing TensorBoard logs\n  logdir = os.path.join(\"./logs\",\n                        # Make it so the logs get tracked whenever we run an experiment\n                        datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n  return tf.keras.callbacks.TensorBoard(logdir)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Early Stopping Callback\n\n[Early stopping](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping) helps prevent overfitting by stopping a model when a certain evaluation metric stops improving.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create early stopping (once our model stops improving, stop training)\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\",\n                                                  patience=3) # stops after 3 rounds of no improvements","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training a model (on a subset of data)\nOur first model is only going to be trained on 1000 images. Or trained on 800 images and then validated on 200 images, meaning 1000 images total or about 10% of the total data.\n\nWe do this to make sure everything is working. And if it is, we can step it up later and train on the entire training dataset.\n\nThe final parameter we'll define before training is `NUM_EPOCHS` (also known as **number of epochs**).\n\n`NUM_EPOCHS` defines how many passes of the data we'd like our model to do. A pass is equivalent to our model trying to find patterns in each dog image and see which patterns relate to each label.\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check again if GPU is available (otherwise computing will take a looooonnnnggggg time)\nprint(\"GPU\", \"available (YESS!!!!)\" if tf.config.list_physical_devices(\"GPU\") else \"not available :(\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":" Let's create a simple function which trains a model. The function will:\n* Create a model using `create_model()`.\n* Setup a TensorBoard callback using `create_tensorboard_callback()` (we do this here so it creates a log directory of the current date and time).\n* Call the `fit()` function on our model passing it the training data, validatation data, number of epochs to train for and the callbacks we'd like to use.\n* Return the fitted model.`"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Build a function to train and return a trained model\nNUM_EPOCHS = 100 \ndef train_model():\n  \"\"\"\n  Trains a given model and returns the trained version.\n  \"\"\"\n  # Create a model\n  model = create_model()\n\n  # Create new TensorBoard session everytime we train a model\n  tensorboard = create_tensorboard_callback()\n\n  # Fit the model to the data passing it the callbacks we created\n  model.fit(x=train_data,\n            epochs=NUM_EPOCHS,\n            validation_data=val_data,\n            validation_freq=1, # check validation metrics every epoch\n            callbacks=[tensorboard, early_stopping])\n  \n  return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fit the model to the data\nmodel = train_model()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Checking the TensorBoard logs\nNow our model has been trained, we can make its performance visual by checking the TensorBoard logs.\n\nThe TensorBoard magic function (`%tensorboard`) will access the logs directory we created earlier and viualize its contents."},{"metadata":{"trusted":true},"cell_type":"code","source":"\n%tensorboard --logdir ./logs \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## Making and evaluating predictions using a trained model\n\nBefore we scale up and train on more data, let's see some other ways we can evaluate our model. Because although accuracy is a pretty good indicator of how our model is doing, it would be even better if we could could see it in action.\n\nMaking predictions with a trained model is as calling `predict()` on it and passing it data in the same format the model was trained on."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Make predictions on the validation data (not used to train on)\npredictions = model.predict(val_data, verbose=1) # verbose shows us how long there is to go\npredictions","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the shape of predictions\npredictions.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Making predictions with our model returns an array with a different value for each label.\n\nIn this case, making predictions on the validation data (200 images) returns an array (`predictions`) of arrays, each containing 120 different values (one for each unique dog breed).\n\nThese different values are the probabilities or the likelihood the model has predicted a certain image being a certain breed of dog. The higher the value, the more likely the model thinks a given image is a specific breed of dog.\n\nLet's see how we'd convert an array of probabilities into an actual label."},{"metadata":{"trusted":true},"cell_type":"code","source":"# First prediction\nprint(predictions[0])\nprint(f\"Max value (probability of prediction): {np.max(predictions[0])}\") # the max probability value predicted by the model\nprint(f\"Sum: {np.sum(predictions[0])}\") # because we used softmax activation in our model, this will be close to 1\nprint(f\"Max index: {np.argmax(predictions[0])}\") # the index of where the max value in predictions[0] occurs\nprint(f\"Predicted label: {unique_breeds[np.argmax(predictions[0])]}\") # the predicted label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Turn prediction probabilities into their respective label (easier to understand)\ndef get_pred_label(prediction_probabilities):\n  \"\"\"\n  Turns an array of prediction probabilities into a label.\n  \"\"\"\n  return unique_breeds[np.argmax(prediction_probabilities)]\n\n# Get a predicted label based on an array of prediction probabilities\npred_label = get_pred_label(predictions[0])\npred_label","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Wonderful! Now we've got a list of all different predictions our model has made, we'll do the same for the validation images and validation labels.\n\nRemember, the model hasn't trained on the validation data, during the `fit()` function, it only used the validation data to evaluate itself. So we can use the validation images to visually compare our models predictions with the validation labels.\n\nSince our validation data (`val_data`) is in batch form, to get a list of validation images and labels, we'll have to unbatch it (using [`unbatch()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#unbatch)) and then turn it into an iterator using [`as_numpy_iterator()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#as_numpy_iterator).\n\nLet's make a small function to do so."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create a function to unbatch a batched dataset\ndef unbatchify(data):\n  \"\"\"\n  Takes a batched dataset of (image, label) Tensors and returns separate arrays\n  of images and labels.\n  \"\"\"\n  images = []\n  labels = []\n  # Loop through unbatched data\n  for image, label in data.unbatch().as_numpy_iterator():\n    images.append(image)\n    labels.append(unique_breeds[np.argmax(label)])\n  return images, labels\n\n# Unbatchify the validation data\nval_images, val_labels = unbatchify(val_data)\nval_images[0], val_labels[0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Nailed it!\n\nNow we've got ways to get:\n* Prediction labels\n* Validation labels (truth labels)\n* Validation images\n\nLet's make some functions to make these all a bit more visualize.\n\nMore specifically, we want to be able to view an image, its predicted label and its actual label (true label).\n\nThe first function we'll create will:\n* Take an array of prediction probabilities, an array of truth labels, an array of images and an integer.\n* Convert the prediction probabilities to a predicted label.\n* Plot the predicted label, its predicted probability, the truth label and target image on a single plot."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_pred(prediction_probabilities, labels, images, n=1):\n  \"\"\"\n  View the prediction, ground truth label and image for sample n.\n  \"\"\"\n  pred_prob, true_label, image = prediction_probabilities[n], labels[n], images[n]\n  \n  # Get the pred label\n  pred_label = get_pred_label(pred_prob)\n  \n  # Plot image & remove ticks\n  plt.imshow(image)\n  plt.xticks([])\n  plt.yticks([])\n\n  # Change the color of the title depending on if the prediction is right or wrong\n  if pred_label == true_label:\n    color = \"green\"\n  else:\n    color = \"red\"\n\n  plt.title(\"{} {:2.0f}% ({})\".format(pred_label,\n                                      np.max(pred_prob)*100,\n                                      true_label),\n                                      color=color)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# View an example prediction, original image and truth label\nplot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Nice! Making functions to help visual your models results are really helpful in understanding how your model is doing.\n\nSince we're working with a multi-class problem (120 different dog breeds), it would also be good to see what other guesses our model is making. More specifically, if our model predicts a certain label with 24% probability, what else did it predict?\n\nLet's build a function to demonstrate. The function will:\n* Take an input of a prediction probabilities array, a ground truth labels array and an integer.\n* Find the predicted label using `get_pred_label()`.\n* Find the top 10:\n  * Prediction probabilities indexes\n  * Prediction probabilities values\n  * Prediction labels\n* Plot the top 10 prediction probability values and labels, coloring the true label green."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_pred_conf(prediction_probabilities, labels, n=1):\n  \"\"\"\n  Plots the top 10 highest prediction confidences along with\n  the truth label for sample n.\n  \"\"\"\n  pred_prob, true_label = prediction_probabilities[n], labels[n]\n\n  # Get the predicted label\n  pred_label = get_pred_label(pred_prob)\n\n  # Find the top 10 prediction confidence indexes\n  top_10_pred_indexes = pred_prob.argsort()[-10:][::-1]\n  # Find the top 10 prediction confidence values\n  top_10_pred_values = pred_prob[top_10_pred_indexes]\n  # Find the top 10 prediction labels\n  top_10_pred_labels = unique_breeds[top_10_pred_indexes]\n\n  # Setup plot\n  top_plot = plt.bar(np.arange(len(top_10_pred_labels)), \n                     top_10_pred_values, \n                     color=\"grey\")\n  plt.xticks(np.arange(len(top_10_pred_labels)),\n             labels=top_10_pred_labels,\n             rotation=\"vertical\")\n\n  # Change color of true label\n  if np.isin(true_label, top_10_pred_labels):\n    top_plot[np.argmax(top_10_pred_labels == true_label)].set_color(\"green\")\n  else:\n    pass","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_pred_conf(prediction_probabilities=predictions,\n               labels=val_labels,\n               n=9)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's check a few predictions and their different values\ni_multiplier = 0\nnum_rows = 3\nnum_cols = 2\nnum_images = num_rows*num_cols\nplt.figure(figsize=(5*2*num_cols, 5*num_rows))\nfor i in range(num_images):\n  plt.subplot(num_rows, 2*num_cols, 2*i+1)\n  plot_pred(prediction_probabilities=predictions,\n            labels=val_labels,\n            images=val_images,\n            n=i+i_multiplier)\n  plt.subplot(num_rows, 2*num_cols, 2*i+2)\n  plot_pred_conf(prediction_probabilities=predictions,\n                labels=val_labels,\n                n=i+i_multiplier)\nplt.tight_layout(h_pad=1.0)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training a model (on the full data)\n\nNow we know our model works on a subset of the data, we can start to move forward with training one on the full data.\n\nAbove, we saved all of the training filepaths to `X` and all of the training labels to `y`. Let's check them out."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Turn full training data in a data batch\nfull_data = create_data_batches(X, y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Instantiate a new model for training on the full dataset\nfull_model = create_model()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create full model callbacks\n\n# TensorBoard callback\nfull_model_tensorboard = create_tensorboard_callback()\n\n# Early stopping callback\n# Note: No validation set when training on all the data, therefore can't monitor validation accruacy\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"accuracy\",\n                                                             patience=3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To monitor the model whilst it trains, we'll load TensorBoard (it should update every 30-seconds or so whilst the model trains)."},{"metadata":{"trusted":true},"cell_type":"code","source":"%tensorboard --logdir ./logs\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fit the full model to the full training data\nfull_model.fit(x=full_data,\n               epochs=NUM_EPOCHS,\n               callbacks=[full_model_tensorboard, \n                          full_model_early_stopping])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Saving and reloading a model\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# creating a function to save the model to use it in the future without training\ndef save_model(model, suffix=None):\n  \"\"\"\n  Saves a given model in a models directory and appends a suffix (str)\n  for clarity and reuse.\n  \"\"\"\n  # Create model directory with current time\n  modeldir = os.path.join(\"./\",\n                          datetime.datetime.now().strftime(\"%Y%m%d-%H%M%s\"))\n  model_path = modeldir + \"-\" + suffix + \".h5\" # save format of model\n  print(f\"Saving model to: {model_path}...\")\n  model.save(model_path)\n  return model_path","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# creating a function to reload a trained model \ndef load_model(model_path):\n  \"\"\"\n  Loads a saved model from a specified path.\n  \"\"\"\n  print(f\"Loading saved model from: {model_path}\")\n  model = tf.keras.models.load_model(model_path,\n                                     custom_objects={\"KerasLayer\":hub.KerasLayer})\n  return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save model to file\nsave_model(full_model, suffix=\"all-images-Adam\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Load in the full model\nloaded_full_model = load_model('./20210104-08491609750160-all-images-Adam.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Making predictions on the test dataset\nTo make predictions on the test data, we'll:\n* Get the test image filenames.\n* Convert the filenames into test data batches using `create_data_batches()` and setting the `test_data` parameter to `True` (since there are no labels with the test images).\n* Make a predictions array by passing the test data batches to the `predict()` function."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Load test image filenames (since we're using os.listdir(), these already have .jpg)\ntest_path = \"../input/dog-breed-identification/test/\"\ntest_filenames = [test_path + fname for fname in os.listdir(test_path)]\n\ntest_filenames[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# How many test images are there?\nlen(test_filenames)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create test data batch\ntest_data = create_data_batches(test_filenames, test_data=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Make predictions on test data batch using the loaded full model\ntest_predictions = loaded_full_model.predict(test_data,\n                                             verbose=-1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check out the test predictions\ntest_predictions[:10]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Preparing test dataset predictions for Kaggle\n\nLooking at the [Kaggle sample submission](https://www.kaggle.com/c/dog-breed-identification/overview/evaluation), it looks like they want the models output probabilities each for label along with the image ID's.\n\nTo get the data in this format, we'll:\n*   Create a pandas DataFrame with an ID column as well as a column for each dog breed.\n*   Add data to the ID column by extracting the test image ID's from their filepaths.\n* Add data (the prediction probabilities) to each of the dog breed columns using the `unique_breeds` list and the `test_predictions` list.\n* Export the DataFrame as a CSV to submit it to Kaggle.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create pandas DataFrame with empty columns\npreds_df = pd.DataFrame(columns=[\"id\"] + list(unique_breeds))\npreds_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Append test image ID's to predictions DataFrame\ntest_path = \"../input/dog-breed-identification/test/\"\npreds_df[\"id\"] = [os.path.splitext(path)[0] for path in os.listdir(test_path)]\npreds_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Add the prediction probabilities to each dog breed column\npreds_df[list(unique_breeds)] = test_predictions\npreds_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds_df.to_csv(\"./full_submission_1_mobilienetV2_adam.csv\",\n                 index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}